For all the industry buzz about the term “Big Data” few know what it entails. So what exactly is Big Data and how does it impact on everyday information security professionals?
Put simply, it's the term used for describing a collection of data sets so large and complex that they can be difficult to process using existing database management tools and traditional data processing applications. Big Data has often allowed for precisely targeted advertising, or real-time analysis of financial trends, for example.
It's a big business, according to research outfit IDC. The firm has forecast that the market will grow at a 40 percent compound annual growth rate from £1.9 billion in 2010 to £10.3 billion by 2015, and this is perhaps unsurprising when other analysts estimate that as many as 200 billion objects will connect to the internet by 2020.
Big Data is not a hard and fast solution, but rather a piece of technology that is increasingly being entwined with security analytics tools and open-source platforms.
It's easy to see why. These solutions harvest Big Data to enable CISOs and IT managers to extract meaning from exabytes of seemingly meaningless data, across various platforms and devices, and almost in real time, too. It can also help dive back in time to look at the detail of events that have happened in the past, or even attacks that are ongoing, such as advanced persistent threats (APTs).
And whereas a conventional analytics platform may only trigger an alarm after three failed login attempts on a critical platform, Big Data tools can do the same while also stipulating commands based on working hours, the employee's job responsibility and the device being used.
“At its core, Big Data is about the ability to extract meaning from massive volumes of disparate data,” says Rashmi Knowles, chief security architect for the EMEA at RSA.
That's not to say that IT managers won't face difficulties, however. After all, Big Data chews up more processing power, and requires more scalability and storage than legacy systems. Budgets are more likely to be stretched too by such solutions, and new issues will arise on data storage, especially in relation to how stored data ties in with internal and external compliance.
It's perhaps unsurprising then that this bandwagon remains in its infancy as far as security analytics is concerned. In fact, some industry observers are unsure if Big Data security analytics represents an extension or replacement of the SIEM solutions most often used for controlling compliance, perimeter monitoring and analysing and aggregating logs, albeit after the event.
Most observers do, though, concur that Big Data security analytics is the next level up from the SIEM solutions that are often overrun with data and unable to embrace newer technologies, like domain controllers and proxy servers connecting to the network. One bank reportedly saw the number of its security events grow from 44 billion a month to 65 billion a month in the space of a year.
Meanwhile, some say that companies are customising open-source tools, like Hadoop, for Big Data security analytics purposes, while others argue that Big Data analytics has been around for years – it's only now that meaning can now be derived from the endless flow of data in a matter of minutes.
Whatever the view, the trend looks set to impact security professionals in a meaningful way in the year ahead as the advantages become more widely recognised.
“The reality is that – whether they realise it or not – almost every organisation has already been hacked,” Ashvin Kamaraju, VP of product development at Vormetric, recently told SC Magazine UK.
“In 2014, enterprises must create security measures that assume hackers are already inside the network perimeter,” Kamaraju says. “Using Big Data for security intelligence will not just be a ‘nice to have' but rather a ‘must have' in 2014. Given all the recent data breaches, we are going to see a rush toward the continuous gathering of security intelligence so that anomalous patterns will bubble to the surface quickly and organisations can respond in near real-time to any perceived threats.”
Further, providing intelligence through Big Data analytics can assess risk, detect problems and interrupt users attempting unsafe activities, says RSA's Knowles. “Analysts will be able to conduct deep-dive investigation in a matter of minutes, where it would normally take hours or days, with more context and clarity than before.”
Security and compliance issues
This, of course, isn't the first time that security analytics has cropped up as an issue in light of the emergence of Big Data.
Back in September of last year, the Cloud Security Alliance's own Big Data Working Group released its yearly “Big Data for Security Intelligence” report, a study which outlined how the landscape of security analytics is changing with the introduction of Big Data tools.
“The goal of Big Data analytics for security is to obtain actionable intelligence in real time,” Alvaro Cardenas, lead author of the report, said at the time. “Although Big Data analytics holds significant promise, there are a number of challenges that must be overcome to realise its true potential. We have only just begun, but are anxious to move forward in helping the industry understand its potential with new research directions in Big Data security.”
But, for all the clamour to piggy-back onto Big Data security analytics, there are some hurdles to overcome. There have been concerns over storage and compliance in particular.
Phil Cracknell, who recently left TNT Express to join Company 85 as head of security and privacy services, believes that the move to cloud services is becoming the facilitator for Big Data security analytics. “Big Data security analytics is about all events occurring on one system, and combining seemingly unrelated results into something meaningful,” he recently told SC.
But Cracknell is concerned that the cloud, while enabling the desired scalability, performance, access, speed and low cost, may lack the necessary controls for enterprise to keep data secure. “Analytics tools do need to catch up,” he says. Companies don't have accessibility to controls through Amazon and other cloud providers as part of their service right now, he points out.
The fact is, Big Data security analytics is essentially “SIEM on steroids,” he says, adding that migration from enterprises to private, hybrid and ultimately public cloud will take SIEM to another level.
“We battled for years with disparate systems within our enterprise to get them all reporting to a central location so that we could utilise this combined data, and now the boundaries have moved outwards and that battle is being fought again, but this time across cloud-based platforms,” he says. The only advantage this time is that because of the cloud, these disparate data streams follow a consistent format so that they can interact from a business operations perspective, he adds.
But Cracknell also has concerns about how data is stored. He is especially concerned with the integrity and confidentiality of data. “It shouldn't be concerning, but we have languished behind perimeters for so long that we have forgotten the basics: Protect the data in its most basic component,” he explains. “If data is protected and classified accordingly, the concerns about pushing it to the cloud lessen.”
In the case of data that has been stored for years, he says security pros must ensure that its integrity is intact and that it has not been modified. “That's a massive challenge for providers.”
Cracknell adds that there are blurred lines, too, on just who is responsible for breach notifications, or notifying users in the case of government eavesdropping.
Bob Tarzey, analyst and director at IT security consultancy Quocirca, agrees with Cracknell that data protection could become an issue. “So, for example, telephony operators are expected to keep records of all calls and this data is also confidential and individual records could be of interest to hackers, whereas the geophysical data of an old company may be high in volume but hard to make sense of at a granular level without context.”
The real issue is understanding what is happening to confidential and regulated data in a sea of less important stuff, he told SC. “This is the realm of data loss prevention more than SIEM, although the latter may have a role to play in clean up or prevention of a data leak.”
Sourcefire's chief scientist Zulfikar Ramzan warns though that as the trend is still in its earliest phase, many companies are still working out the capabilities of Big Data in relation to security analytics. “We're still in the early phases,” he says. “This is just the tip of the iceberg. There's a popular misconception that Big Data is a magic wand, but it's not just one model. There may be different solutions to different problems.”
BIG DATA: The big picture
Bob Tarzey, analyst and director at IT security consultancy Quocirca, says there are two elements of Big Data: “Security from Big Data” (i.e., ensuring these huge volumes of business data are treated with appropriate levels of security) and “Security of Big Data” (how such solutions can be used to improve analytics).