Irisscon: Annual penetration tests on applications leave you unsecure

News by Dan Raywood

A major window of exposure is created by annual penetration testing.

A major window of exposure is created by annual penetration testing.

Speaking at the Irisscon event in Dublin, the annual conference for the Irish reporting and information security service (IRISS) and computer emergency readiness team (CERT), Eoin Keary, director at BCC Risk Advisory, said that "everything we do in order to secure web application is wrong in my view", as major brands continue to be hacked, "so there must be something wrong".

Keary called it an "asymmetric arms race" as attackers look for low hanging fruit, while businesses have a traditional approach of annual/biannual penetration testing to tick a box, but that this doesn't really work.

“We have systems that take many man years of development but only a week to penetration test it, a defender has 20 man days per year to detect and defend, who is going to win? Anyone can capture flag and it is about time, and time costs money,” he said.

“Also web applications change over time, businesses update stuff and there is a bit of a problem with that. When you do penetration testing you get false positives or false negatives; when we do it we use numerous tools so it is software testing software and we tend to get various results and tools and it takes time, as in effect you need to customise your attack vectors.

“We need to be at least as good as the bad guys. We are defending against something we cannot possibly do. This is what we do today, and in my view it is completely wrong.”

Keary claimed that after a penetration test, the code is fixed a week later so the window of exposure is a year until the next penetration test. “You can have all the cool tools, but if you don't understand how to use them then you're not going to win,” he said.

“You need continuous monitoring and testing, and vulnerability management. How do we improve something we cannot measure? You need more than a penetration test.”

Evaluating the typical time spent penetration testing, Keary said that if 300 web application tests are done once a year, this involves 1,500 working days and this results in 300 reports.

He said: “So the idea is enterprise security intelligence – data mining and analytics and consolidation of data, also continuous monitoring and [the need to] manage data and what you need to fix and when.

“You need to understand what you are protecting against and that penetration testing annually is a losing battle as hackers do not care, and you can only improve what you can measure.”


Find this article useful?

Get more great articles like this in your inbox every lunchtime

Video and interviews