James Parry, technical manager, Auriga
James Parry, technical manager, Auriga

Threat intelligence: it's the latest buzzword in the security industry and the shiny new solution coveted by CIO's. The theory goes that by adopting a proactive stance, and monitoring activity not just on the network but externally too, you'll have advance warning of an attack. Events or triggers can be spotted that indicate, like ripples on a pond, the approach of a predator, robbing the attacker of the element of surprise and giving the organisation time to raise its guns and throw up the defences.

The trouble with this scenario is that big business has been doing this type of monitoring for some time and with some expensive tools… yet attacks are continuing unabated. Anti-virus, intrusion prevention systems (IPS), data loss prevention (DLP), and Security Incident and Event Management (SIEM) systems are all being used to automatically collate and log data and events in a bid to crunch sufficient data to stymie an attack.

Yet each has varying levels of success. Anti-virus is reported to capture only 25 percent of advanced persistent threats (APTs), for example, and a recent survey found up to half of attacks avoid detection through the use of encryption and the knowledge that most of these systems rely on known signatures.

Perhaps threat intelligence isn't the solution after all. Or perhaps it's just part of the answer? What's rapidly becoming clear is that there's no substitute for human analysis. This is seeing the emergence of a new concept: threat hunting. Threat hunting combines threat intelligence gathering by automated solutions (technical tools and machine learning techniques) with human data analysis to create a more intuitive, responsive and human-led form of threat detection. It's one that treats threat intel as the start of the process, not the end result, and seeks to extrapolate and act on that intelligence.

According to a recent SANS report, threat hunting is a continuous process that seeks to aggressively track what SANS call “indicators of compromise” (IoC) through automated threat detection systems but also, crucially, analysts. It's this human element that is able to turn straw into gold by spotting anomalies, inconsistencies and patterns in the emerging data. That information can then be subjected to analysis and used to interpret and forecast events, something that is out of the question for an automated system.

Threat forecasting essentially escalates known network events, factoring in other variables such as time, sector, data target etc, to determine likely scenarios. It's this ability to contextualise the threats that can make the difference to the business, buying time to protect sensitive data and increase protection. It also allows the application of logic. Simply responding to external intelligence can see time and resource wasted by focusing on the wrong place. Applying sector specific and company specific criteria to the intelligence can help focus efforts and determine the level of response required. It can also allow the team to look at the wider picture and how the attacker may have probed or expanded the attack once a weakness has been detected. 

It's a common misconception that thwarting one attack means the organisation is better positioned to fight off attacks of a similar nature. This does not necessarily follow. Attackers are constantly seeking to improve and adapt and the only means of defending against this type of evolving attack is to adopt real-time threat hunting. This requires the monitoring, aggregation and processing of dynamic data from numerous sources and such automation can only be carried out by a Security Operations Centre (SOC).

A SOC can carry out the level of capture required but it's still no substitute for human intelligence and this is where an outsourced solution comes into its own. Deploying a SOC in-house is common practice among big business but this can cause problems. The small team responsible for managing the SOC won't necessarily have up-to-date skill sets or knowledge of current threat behaviours and what to look for. They often work in isolation, so will be unaware of the cultural signifiers associated with a given sector or geographic market. There are numerous examples of instances where such early warning signals were missed, from the attacks against the hospitality industry and big hotel chains last year, to repeated DoS attacks against the banking sector. All can and should have been detected if threat hunting had been used.

There will still be those who invest in the bells and whistles of an in-house SOC. But there is no substitute for field-based threat intelligence by a human hunter.

Contributed by James Parry, technical manager, Auriga