Product Group Tests

Vulnerability assessment (2009)

Group Summary

Our top award of SC Lab Approved goes to Core Impact Pro 8 for highly specialised and sophisticated end-to-end testing for the enterprise.

An excellent vulnerability scanner at a good price makes Retina Network Security Scanner our Best Buy.

Solid performance and ease of use win DbProtect our Recommended award this month.

Scroll To Full Group Summary Below

Click for a side by side comparison of products
Click for a side by side comparison of products

Full Group Summary

The security assessment field is maturing. We put 12 products to the test. By Peter Stephenson.

This was an interesting year for our vulnerability assessment (VA) group test. Last year, we separated application vulnerability assessment from network vulnerability assessment. This year we grouped them together. This revealed a few interesting differences.

The primary difference is that network vulnerability assessment tools are converging with penetration testing tools to provide both capabilities in the same device. This is important because, properly used, penetration testing is an extension of vulnerability assessment. In a proper network security assessment, one begins with the large view and progresses towards the specifics. Two years ago, there were no solid combination tools. Last year, we had a couple that got pretty close. This year, we had solid entries that are really single security assessment devices.

We differentiate between vulnerability assessment, penetration testing and security assessment. Vulnerability assessment (VA) reveals the global picture of possible vulnerabilities. I say "possible vulnerabilities", because VA tools can give false positives and, sometimes, the existence of a vulnerability does not constitute a risk. In order to have a risk, the vulnerability must be reachable by a threat and there must be a threat to exploit it.

We have learned, perhaps to our sorrow, that when there is a vulnerability we should pay attention to it. With SQLSlammer and now Conficker we have exploits of vulnerabilities that were announced and had patches provided, months before an exploit appeared. So, when we identify a vulnerability, we need to test it thoroughly to determine if it can contribute to a risk. We do that by focusing on potential vulnerabilities with a tool to attempt to exploit the vulnerability: penetration testing.

Having all the tools, from discovery and foot printing through VA to pen testing, in one network security assessment product, is very useful. This offers consolidated reporting, simplified point testing and easier compliance testing and reporting. Unfortunately, application VA tools are not quite that mature yet.

However, given what we saw this year, that maturation is not far off. Perhaps we'll see the same kind of consolidated testing in applications as we do in networks. We will know security assessment has come to full maturity when all the tools are converged into a single one. That cannot be far off.

Even though substantive changes were few, one thing we did observe was that as the pack chases the perennial leaders, the gaps at the front are narrowing. One product in particular was a very pleasant surprise. We have been watching this for years and it is up towards the top this year.

The second important trend we are seeing is a slow move to vulnerability management. That means that some products are focusing more on remediation than in the past. Products in this class have always offered remediation recommendations, but now we are beginning to see devices that actually help by providing assistance in patching. While VA products have yet to reach the level of patch management tools, we expect to see this evolve.

Compliance is probably the number one driver in information assurance at the moment. VA tools are responding by providing scan and testing templates that address specific regulatory requirements. This is an interesting twist on many other kinds of products that test or manage systems generally and then create specialised compliance reports.

The jury is out on this approach, because if you need to look at a comprehensive set of possible vulnerabilities, it is good to test completely and refine at the reporting stage. There is the unfortunate potential to support the "tick-in-the-box" syndrome - when an organisation opts to make sure that it can defend the checkmarks on the audit report, instead of making the enterprise truly secure.

The bottom line is that the security assessment field is maturing. Tools are addressing user needs instead of addressing point solutions. The buzz-phrase must be "vulnerability management" instead of, simply, "vulnerability assessment" or "penetration testing".

Security assessment emphasises the holistic nature of understanding the weaknesses in the enterprise. It goes beyond testing to include security assessment, VA, pen testing and patch management - all in a single device. This new class of products is going to become an integral part of the overall enterprise management infrastructure.

All Products In This Group Test