Effective incident response: not as easy as it seems!
Effective incident response: not as easy as it seems!
Most organisations understand the threat of cyber-attack and are taking steps to prevent and detect attackers in the network. But while hundreds of security vendors are working to build new technologies to help discover imminent attacks, there are far fewer options on the market for helping security analysts to effectively respond to those attacks that are detected. Estimates suggest that, on average, it takes 50 to 60 days for security teams to contain and respond to incidents. This means 50 to 60 days where the attacker has the opportunity to move further into the network, exfiltrate data or create a new user profile that won't have its password reset. 

While most security professionals will admit that it is relatively straightforward to complete the data-gathering phase of Incident Response (IR), putting the data together in a useful way, and doing this quickly, is actually quite a challenge. But, until you can make sense of all the available information, you can't shut down the attack. In practice, there are four common challenges that prevent IR teams from responding effectively and efficiently to threats:

Lack of useful logs
It should be obvious that if the forensics information doesn't exist, you can't do much with it. Yet, it's surprising how often organisations simply don't log the critical information that they would need to conduct an effective incident response. For example, if a business only logs failed logon attempts, they would have no way of tracking attackers who enter the network using compromised, but legitimate, credentials. 

At a very minimum, for every endpoint, businesses should log both successful and unsuccessful logons, changes or additions to user or group accounts, process creation and termination, and PowerShell logs. On the network side, they should log DNS queries, proxy logs, and NetFlow information, as these are valuable historical data sources that could make or break an IR process.

Accessing information at scale
Some information is useful, but very complicated to access at scale. For example, in smaller investigations, it might be necessary to have access to a full disk image of a user's workstation, to look for malware or other indicators of compromise. In larger investigations, it may be that the threat researcher needs to look on every employee's machine for those same indicators. Getting a disk image from one machine isn't hard; getting it from 50,000 endpoints may be impossible and would result in way more information than is needed to answer the relevant questions and find the threat. 

The best way to overcome this challenge is to use some kind of centralised logging, which can make the process much easier to scale. Other endpoint technologies such as Carbon Black, osquery, and Mozilla InvestiGator can also help to gather information needed for IR across a large number of endpoints.

Having the right security talent
Many businesses simply don't have the bodies and connected brains needed to investigate and analyse an incident properly. This may be due to frozen staffing budgets or simple inability to hire what's needed. With The Telegraph reporting that the global shortfall in cyber-security experts is expected to increase 20 percent to 1.8 million by 2022, this problem will only intensify over the next few years. In practice, this means that when an incident hits, it's too slow or not even possible to investigate using the available people. 

The usual answer to this challenge is to hire forensic consultants, but this isn't always possible or practical. The skills shortages are impacting forensic consultancies too; they also face shortages and may not be able to staff a project in time to contain an incident. This means that consultancies have become a very expensive option. 

The shortage of people is a really tough problem, as you can't create new experts overnight. The answer will lie in automation, which can be used to amplify and guide the security analysts that businesses already employ. It's now possible to automate data gathering, timeline creation, reputation and context, making life easier for analysts and cutting response times dramatically. It can also make employees more efficient (and happier!) by eliminating some of the tedious, repetitive parts of an incident investigation.

Team collaboration 
The volume and complexity of incidents today mean that the old ways of sharing information no longer work well. IR teams typically track notes and data in a shared document and discuss the information over instant messenger. The problem is, in most post-incident scenarios, there will be multiple forensics experts involved in the IR. These people could be in the same location, but work in different shifts to provide around-the clock remediation. They might even work in different locations at different times.

When the day-time IR analysts go home, the next shift comes in and they need to be able to see exactly what their colleagues have discovered. When this information resides solely in a spread sheet, it makes it harder to hand over to the next team. The handover process can slow down the IR, or the analysts run the risk of missing something and not responding effectively at all. For example, an analyst could accidentally erase some key information if it isn't properly recorded that another analyst found something of interest within that data.

Thankfully, dedicated tools are coming to market that help IR teams collaborate, share notes, and respond more quickly. These tools provide a notebook function that is shared and updated in real time and all information can be time stamped to create a forensics time line and make sure that handovers are smooth. 

Overcoming the barriers
Some of these challenges are likely to continue for the foreseeable future: it's hard to find good people, and even harder to coordinate them. Changing processes or adopting new technologies can overcome some issues. The bottom line is that we need to get better at managing data at scale, at automating the tasks that slow down analysts, and at amplifying those analysts' abilities to make incident response as effective as possible.  

Contributed by Ryan Benson, senior threat researcher at Exabeam

*Note: The views expressed in this blog are those of the author and do not necessarily reflect the views of SC Media or Haymarket Media.