More than three-quarters of Europe's largest Big Data community believe that the UK has a shortage of specialists in the area.
According to a survey of 131 members of the ‘Big Data London' group, who are primarily people working on Big Data projects, 77.9 per cent of respondents believe there is a shortage, while 70 per cent think that there is a knowledge gap between the skilled Big Data workers and those commissioning projects.
The survey also found that more than three-quarters of respondents say learning ‘on the job' (80 per cent) and ‘self-teaching' (72 per cent) are the best ways of keeping up to date with the latest Big Data skills.
Manu Marchal, director of Acunu, who conducted the survey and is Big Data London organiser, said that there is the potential for problems in the UK if companies don't respond to the skill shortage.
“It's easy to look at a skills shortage and blame a lack of training on traditional educational routes, but when you're on the cutting edge of technology, you have to be teaching yourself most of the time,” Marchal said.
Talking to SC Magazine earlier this year, Splunk claimed that there is a shortage of skilled data analysts. Its director of technical services in the EMEA region, DJ Skillman, said: “This is a challenge people will run into, managing more data. A lot of developers know how to develop, but people are coming up with an algorithm and asking the questions correctly, but the process is wrong. One of the challenges we come up against with Big Data is knowing how to process.”
Terry Neal, CEO of InfoSec Skills, said: “We need a new breed of security analyst - a data scientist, who can integrate emerging frameworks to data mining and predictive analytics disciplines to maintain the battle front against cyber crime. The problem is that organisations may not have the internal analytics skills and the cost of hiring experienced analytics professionals is high.”
When asked about the two main problems of understanding Big Data, the most popular answers were ‘under utilisation of Big Data' (66 per cent) and ‘having unrealistic expectations for Big Data' (66 per cent).
Marchal said: “Both answers are reactions to incomplete knowledge. Big Data has been talked about for what feels like a long time, but the reality is that this technology is constantly evolving so management, like practitioners, are often not aware what can be achieved with these technologies.”