Humans are 'better than AI' at discovering vulnerabilities - say humans

News by Mark Mayne

A new research survey claims that humans are still in the driving seat of vulnerability research when compared to ML/AI tools - AI yet to claim a bug-bounty

The vast majority of security experts believe that human pen testers and white-hat hackers are more effective at finding vulnerabilities, according to a survey from HackerOne. 

Indeed, 53 percent of security professionals believe that hackers and pentesters win over automated tools, with a mere 27 percent betting entirely on machines. 

Laurie Mercer, security engineer at HackerOne told SC Media UK that uncovering new vulnerabilities was not as easy for AI as might be imagined: "Researchers have long been fascinated with the idea of using AI engines to look for vulnerabilities, and several approaches have been tried already. If AI can recognise a face, then it should be able to recognise a vulnerability. 

This decade AI has proved that once it has learnt the rules of the game it can beat humans at both Chess and Go, and is on its way to operate cars safer than a human driver. The problem with hacking and AI is that the whole point is to break the rules. The definition of 'to hack' is to 'creatively overcome obstacles', and it is this ability to break or define their own rules that elude modern AI algorithms.

"Having said that, rumours abound that the singularity is near. AI absolutely has the potential to find vulnerabilities, and we look forward to the day when AI earns its first bounty."

The study also found that more than one in ten (12 percent) of organisations had suffered a security breach as a result of an unpatched vulnerability and that awareness was high - 79 percent of respondents said they thought unknown security vulnerabilities posed a serious threat to their organisation.

Matt Walmsley, EMEA Director at Vectra said: "Human creativity, intuition and contextual understanding remains at the very heart of good cyber-security practice and staying current with your patching is a key task to reducing your attack surface.

Whilst the vast majority of security research is still human led, there are increasing opportunities for automation to move beyond simply scanning for previously known vulnerabilities, towards more automated "red team" penetration testing.  The "blue team" defenders too are increasingly using automation, often powered by AI to spot known and unknown threats based upon the immutable behaviours that signal active attacks, without the need for prior knowledge of the tools, exploits, and vulnerabilities they are using.

There are no perfect systems, as a profession we must continue to research and identify and ethically share vulnerabilities and build detection and response capabilities that don’t rely on prior knowledge of the threats."

The research, carried out at Infosecurity Europe 2019, also found that organisations were keen to work with hackers to identify vulnerabilities (63 percent), but using hackers for constant monitoring or to recruit new talent were much less popular, at 14 percent and 10 percent respectively. Interestingly, although most (91 percent) of security professionals believe hackers should be rewarded for finding vulnerabilities, only 63 percent believe hackers should only be  rewarded if they follow the correct vulnerability disclosure process…

Find this article useful?

Get more great articles like this in your inbox every lunchtime

Video and interviews

Interview - Everyone has an Achilles heel: The new security paradigm

How can we defend networks now that the perimeter has all but disappeared?
Brought to you in partnership with ExtraHop