The reality is that pen-testing provides no guarantees of security, and does not address the weaknesses in an organisation's ability to detect and respond to a sophisticated attack; or its ability to manage a cyber crisis and take the timely decisions to enact cyber defence or system continuity plans.
Consequently, this is driving the need for more sophisticated and technically-based crisis exercises to identify causes of failure.
To most firms, a real-world attack simulation is as much a 'game changer' as actually being targeted. In both cases, firms can expect to learn hard lessons but the war game process ensures that the organisation is ready to absorb the lessons, and identify the benefits without the pain or damage of an actual breach. This point cannot be underestimated. In a real event there is invariably a catalogue of human and management failures consistent with the inability to think clearly under pressure.
In reality, most lessons are only learnt after a real event, even when the overriding climate is negative or less orientated towards learning. A war-game, which simulates a prolonged attack, aims to provide lessons before a real event, and enables learning during an attack. In short, it can develop a firm's ability to interpret and apply experience into real-time learning.
Cyber war games derive significant learning across multiple levels of decision-makers, and can be structured specifically to bring together the CISO, security leadership team, security operations centre, incident response, as well as the forensics, risk, and crisis management teams.
War gaming is an excellent and effective way for large organisations to identify the weaknesses in communications and coordination between these groups. In times of crisis, the cascading effects of an attack and the impacts are often exacerbated by the decisions taken, and the process of decision-making by these groups. Learning how these groups take certain decisions when faced with uncertainties, or adapt and enact response plans when tackling ‘unknowns' is vital to successful response.
Cyber war games are new and are slowly being adopted because there are currently few bodies providing the scope of capabilities required to conduct such an exercise. Elements to be considered include the set-up of both internal and external directorates, the preparation of the 'red team', and any required custom tools are necessary to move away from a one-dimensional desktop approach. To prepare an organisation's ‘blue team' with the appropriate preparation may even require a pre-exercise review of all of the following, depending on the objectives set: policies and procedures – the gap is measured against best practices, employed methodologies, deployed technologies, and past lessons learned.
A well-crafted war game incorporates both a ‘fundamental surprise' that the organisation had not anticipated and a number of ‘situational surprises' which were known cyber risks for which the organisation has little or no advanced warning.
Much of the pre-exercise planning should aim at developing appropriate knowledge and intelligence in order to define the exercise in a manner that can be controlled and developed over time, and tests the different capabilities.
The ‘storyline' can commence with a technical event to kick off the assessment of initial implications, and the event would then be developed through situational feeds from the directorate.
The initial objectives should be to test detection: by the systems; by the incident response team; and the analysis of the forensic team. More can then be provided by the directorate including intelligence, such as analysis of the threat community, IP information, and pieces of a malware. The exercise can then examine the fundamentals of communication and decision-making, specifically who is taking decisions and on what basis; and what is the process of taking alerts/indications and deriving useful information from then: and then transforming that information into knowledge throughout this first technical phase.
At this point, a major new technical event may be introduced, or the original event may be taken in a new direction to trigger a new cycle of detection and decision-making. Evaluation may focus more on how the new event affects the decisions previously taken, the need for additional resources, and whether a new risk assessment should take place. With a second phase escalation of the attack, the evaluation can examine who is assessing the risk throughout the event, who is involved in the process, what indicators are in place, and how they conduct a timely assessment of the possible implications from the new event.
Using this approach will allow escalation towards the involvement of the crisis management team, and an examination of their team, what stage they were involved and how they receive the relevant information. The exercise can also test the team's communication effectiveness, who precisely was evolved and how they supported the whole process.
The more significant element in the learning process is the incorporation of observation, decision-logging, and mentoring as part of the war game process, while a full debrief and post exercise workshop should establish lessons learnt, capability gaps and the modifications required in technology and processes.
It is advised that a full day is then allocated to analyse all events, and outcomes of the exercise, reviewing performance of the different groups, and the effectiveness of deployed the technology. The teams involved should be encouraged to appraise the effectiveness of work process, and develop lessons to be learned with the observers and mentors.
The 'learning by doing' opportunity that war games provide identifies failures in breach incident response as well as failures in security.
This should ensure a balance between security and implementing the appropriate response, but also offer a list of immediate tactical priorities for remediation, as well as short term changes. It can also pick up previously peripheral issues that had not been addressed or prioritised specifically because they may have been proven to be more critical to the overall security apparatus than previously recognised. Often these are 'human' aspects known to be weaknesses, though not recognised and addressed at an organisational level.
By establishing the right war game framework, particularly because a technical attack process in central, the learning objectives are set at the top of the agenda if the organisation is astute enough to accept that a breach will occur, and the success is measured by how it deals with this.
To support this shift in perspective, end-of-exercise workshops can be used to help participants understand what was previously lacking, and provide the opportunity to build consensus around priorities from board level down through the risk, business continuity, and security teams.
The iterative process of this type of workshop can offer a forum for planning that integrates investment, and priorities between prevention, defence, and a shared understanding of the converged nature of cyber risk. This pre-emptive approach to develop effective cyber defence and identifying causes of future failure identifies priorities for response training, and the development of a response doctrine that can provide agility and options.
Dan Soloman heads the Cyber Risk and Security Services division at Optimal Risk