The evolution of analytics in  threat detection
The evolution of analytics in threat detection

Almost on a daily basis, we hear that a cyber-security breach has exposed the data of millions of customers, or of yet another major company's scramble to contain the damage of a cyber-attack. As we collectively groan that our private info has been exposed yet again, it's clear that traditional security is no longer enough to protect enterprise data and systems.

The methods of skilled, motivated, and well-funded attackers continue to evolve over time. Fortunately, so are those of the defenders. While no single solution blankets the many facets that span information security today, professionals understand the key to successful detection and response: mastery over one's data.

Traditional countermeasures

Most security professionals are familiar with the traditional, preventative ways of blocking malware. These tools detect malware by matching endpoint data against known-bad signatures.

In this method, the code making up each program on a computer system is represented as a unique string of characters—a signature—called a hash. For example, each particular version of chrome.exe can be represented by an algorithmically derived hash value much smaller than the program itself. Reducing known malware to these tiny, byte-sized hashes allows detection software to maintain a blacklist of known “bad programs” and stop any “matches” from running.

In the past, signature detection relied on users to update local copies of the blacklists, or signature files. These days, this information is centralised in the cloud, a notable step forward. Instead of relying on users to manually get fresh threat indicators, anti-malware software continually checks end user files against the latest and greatest list.

While this important upgrade improves the effectiveness of signature detection, this approach presents two major challenges. First, it requires researchers to spot new malware in the wild and add it to a database. Second, increasingly sophisticated developers can now manipulate malware to appear harmless.

For these reasons and more, signature-based detection, while still an important part of the security toolkit, cannot provide comprehensive coverage.

Modern incident detection and response

All of today's best approaches start with thorough data collection, and then running layers of analytics to expose threats. The idea is to detect “unknown-unknowns”—attacks that have never been seen before—as well as detect compromise that doesn't require malware to be successful. Comprehensive data collection, typically done by security information and event management (SIEM) tools, is now the foundation of the detection castle.

Traditionally, SIEM solutions have been most useful during incident investigations as the centralised location for log files and security events. Despite this wealth of data, SIEM struggled to meaningfully surface malicious behaviours, as the burden of writing and tuning detection rules was left to the customer, straining security teams.

Detection rules may alert on the following:

●     A company employee authenticates outside of an office location (eg Russia).

●     A non-Finance employee authenticates to the Finance server.

●     A user tries a bad password ten times.

While each rule can identify malicious behaviour, they can also come with time-consuming noise. Employees might be travelling, have special privileges for a project, or are just having a terrible keyboard day.

This is where advanced analytics come in. Graph mining and entity relationship modelling can baseline “normal” relationships between users and assets on the network. This specifically highlights when authentication patterns look like unusual user behaviour or lateral movement. By going beyond logs to directly ingest endpoint data, even more is possible. For example, analysing service creation events can identify abnormal processes being launched remotely. This can detect malicious use of PSEXEC, a built-in IT administration tool borrowed by attackers to reduce their reliance on malware.

With this combination of baselining and focus on user monitoring, today's detections can alert on:

●     An employee logs onto the corporate network. Ten minutes later, that user's cloud credentials are attempted from an “impossible to reach in that time-span” geographic location.

●     An authorised user authenticates to the Finance server, but, from a never-before-seen laptop.

●     An entity tries one password (eg Fall20!8) across every account in Active Directory.

The best advanced analytics should guide analysts through investigation and response. Detection is only a piece of the response workflow—it's important to not only identify the initial attack vector, but each step the attacker took from there. What's on the horizon? Machine learning and even artificial intelligence are being touted as catch-alls, but make sure the claims are backed by security research, or test the tech in your environment. Across machines and humans, one tenet still remains true: “If you don't know what you're looking for, you'll never find it.

To combat these threats, security professionals can deploy honeypots—intentionally vulnerable machines on a corporate network—to gather threat intelligence and identify risky user behaviour. Using a honeypot is as simple as deploying it and monitoring connection attempts from the rest of the network. Most employees don't perform network scans in their day-to-day. If one of their assets starts communicating with the honeypot, this either reveals a misconfiguration or compromise by an outside entity.

Honeypots are therefore a high-fidelity detection mechanism, but they only highlight a small range of behaviours. When investigating an alert generated by a honeypot, it can be very challenging to determine a root cause, unless there are other sources of data to match against. In other words, a honeypot will tell you that something is amiss, but not what to do about it.

Each of the above technologies has merit and can be great at detecting specific malicious behaviours. But in today's threat environment, more is needed for truly effective detection and response.

Contributed by Eric Sun, solutions manager for incident detection & response, Rapid7.

*Note: The views expressed in this blog are those of the author and do not necessarily reflect the views of SC Media or Haymarket Media.