One of the key themes of our look forward to 2012 also became a key talking point of last week's RSA Conference.
A recent survey by LogLogic found knowledge of Big Data – what it actually is – to be restricted, and going by some of the comments made last week, it is hard to determine whether there is a genuine understanding of the challenge, let alone its resolutions.
Security commentator Bruce Schneier said "the rise of Big Data is a threat we need to take seriously" in his presentation, while RSA executive president Art Coviello said in his keynote that Big Data refers to the gathering of security-relevant data sets in unprecedented scale and in numerous formats, which must be gathered from every part of an infrastructure and beyond, and correlated using high-speed analytics to produce actionable information.
Coviello said: “The age of Big Data has arrived in security management, enabled by advances in data storage systems, computing power and analytical tools that, when combined, eliminate the old trade-off between the cost to collect and store data on the one hand and the cost and time required to analyse the data on the other.
“With this Big Data capability, security teams can stop wasting money on obsolete controls and time-tracking those meaningless individual events. They'll have what they really need to be most effective in their jobs, ready answers to the most difficult questions about advanced threats, compliance, fraud and other risks.
“Security teams will have the power to recognise the enemy within quickly, isolate compromised elements of infrastructure, protect information assets and render attacks harmless. In essence, Big Data gives you the power to shrink your window of vulnerability.”
So is this an extension of how security information and event management (SIEM) technology should be used? Richard Bejtlich, chief security officer at Mandiant, said on Twitter that his take on Big Data was that it is "detection = collection + analysis and response = escalation + resolution", and that "too many declare victory after collection".
I recently spoke with Chris Boorman, chief marketing officer at Informatica, about the sensation of Big Data and whether he felt it was just a marketing term to facilitate the sale of data-crunching technology.
He said: “This is something that is happening now, and vendors are jumping on the Big Data bandwagon. I have never seen this happen before, where a company has no idea where data is, and I have never seen such a change. Every organisation is now looking at their environment.
“It is about how you evolve the cloud and use applications and data. If it is now beyond the firewall, it is now hitting the enterprise. Everyone is now reeling under the impact and this is an opportunity of all sorts for new types of data, to understand what it means and how to use it.”
In a recent Forrester blog, James Kobielus asked if Big Data was "marketecture", or referred to "a set of approaches that are converging toward a common architecture that might evolve into a well-defined data analytics market segment".
He said: “When, if ever, will data scientists and others be able to lay their hands on truly integrated tools that speed development of the full range of Big Data applications on the full range of Big Data platforms?
“Perhaps that question is also a bit overbroad. Here's even greater specificity: when will one-stop-shop data analytic tool vendors emerge to field integrated development environments (IDEs) for all or most of the following advanced analytics capabilities at the heart of Big Data?”
He said he doubted that a technology – which would need to include data architecture, data integration, data governance, master data management, metadata management, business rules management, business process management, online analytical processing, dashboarding, advanced visualisation and other key infrastructure components – would emerge any time soon.
“The only vendors whose current product portfolios span most of this functional range are SAS Institute, IBM and Oracle. I haven't seen any push by any of them to coalesce what they each have into unified Big Data tools,” he said.
Another analyst report by Matthew Aslett from the 451 Group said "the biggest problem with Big Data… is that the term has not been – and arguably cannot be – defined in any measurable way" – because the size of the Big Data market cannot be determined. “You may as well ask ‘how long is a piece of string?',” he added.
I put these thoughts to Boorman. He said that the challenge of Big Data has come about because of the transition from the desktop to the mobile device, with this enabling different ways of processing data and new technologies.
He said: “Big Data is a confluence of traditional data and Big Data processing, and organisations will want to know how to do this. More data needs to be integrated and turned into value.
“In terms of it being a marketing term that is in vogue, Big Data is about data being broken down and put into compartments so it can be used better. It is not just about moving and copying data, new data is being created: from GPS off a phone; via social networking sites; it is genomic data from pharmaceutical devices; or smart metering data – all types are affecting enterprises.”
You could call Big Data a marketing term, but in the same vein you could call governance, risk and compliance (GRC), or ‘infosec', terms that have been created by marketing and thought leaders.
Regardless of the label, Big Data has now become such a key factor of IT and information security that it has gained its own name, and while managing large amounts of data has to be taken seriously, managing Big Data has never been so important, complicated, and vital to business security.