Product Group Tests
Vulnerability assessment (2007)
Rapid7 Nexpose is not cheap, but it delivers a lot of bang for the buck and we rate it our Best Buy in the hybrid class. In the scanner-only category, we consider NetClarity’s Branch Auditor 5.0 a Best Buy for its powerful performance, ease of use and excellent documentation.
Saint Scanner + Exploit receives a Recommended award for its useful combination of scanner and penetration tool. We rate Tenable's Passive Vulnerability Scanner Recommended for its powerful approach to compliance and general network vulnerability monitoring, while Tenable Nessus 3 comes Recommended as a second scanner or as part of the Security Center.
We rate Core Impact as Approved for SC Labs for its comprehensive capability in a production environment, performance and ease of use.
Full Group Summary
Vulnerability assessment tools are getting better and better, with some now adding penetration testing to the mix. And the spread of Nessus is adding a further dimension. By Peter Stephenson.
This month we looked at vulnerability assessment and penetration test tools. The main difference when compared to last year's test is that this time round we saw more hybrid products that offered both vulnerability scanning and penetration testing. We also reviewed a passive scanner for the first time and saw a lot more attention being paid to meeting regulatory requirements, especially in the payment card industry.
As always, we had a nice bunch of products that included appliances and software-based solutions. One product was strictly a penetration testing tool and, as we had some other products that combined vulnerability assessment and penetration testing, this forced us to break the group up into three sub-categories. These are vulnerability assessment, penetration testing and hybrid.
Our general observation is that appliances are becoming the platform du jour for this type of product. Generally, the appliances offered more features than software-only options and the reporting was more robust. Additionally, we are seeing more products based on the open- source version of Nessus. We included Nessus in this review because it is one of the most popular vulnerability scanners available. The differentiators for the solutions using Nessus as a core platform are ease of use and available extensions, such as user interface, advanced reporting etc.
Finally, another important trend is distributed scanners reporting to a central console that acts both as a management console and results correlator. These systems usually offer sophisticated management, correlation and reporting capabilities, as well as add-on services such as patch notification, trouble ticketing and remediation assistance.
Hybrid products are not always suitable for all organisations. There are reasons to scan only, reasons to buy a solid penetration testing tool, and reasons to buy a hybrid tool that does both. We see straight vulnerability testing tools as appropriate for ongoing testing. Regular scanning reports can now be integrated into multipurpose devices, SIM/SEM products and the new category of security risk management (SRM) products to give a more complete view of the network's security posture. We recommend that all organisations use some sort of regular vulnerability scanning.
Penetration is appropriate as an add-in to scanning. Scanners will tell you where the holes are likely to be, but penetration testing tools can attempt to exploit those potential holes to let you know for certain. If you have neither, a hybrid is a good bet. Moreover, vulnerability testing is a bit difficult in the sense that scanners often find false positives. Some products recognise a possible false positive and tag it for you. The most reliable approach is to use two scanners or a scanner and a penetration tool.
Penetration by itself is not reliable unless the pen tool performs a preliminary scan to decide where to test. Core Impact, for example, uses this technique if you run all of the scripted test sequences, but it is a penetration tool and cannot be considered to be a scanner. Scanning simply is part of its overall technique to identify possible holes.
How we tested
Our vulnerability test bed consisted of examples of several operating environments, patched and unpatched. Our victim suite included MS Windows XP and 2000, both as shipped and with current service packs and updates, two versions of Linux and Solaris. We took vulnerabilities detected from the scanners and measured the ability to exploit holes for the pen tools. We also ran penetration scripts when available. This technique runs a suite of pen tests based on things such as operating environment, version and patch levels and open services.
From the functional perspective, we looked for ease of use in a production environment where many tests must be run by few people in little time. We wanted the penetration tools to prove that they got into the system by putting a shell agent on the victim or by harvesting or placing files.
We evaluated the number and type of reports, whether custom reports are possible, and how the product presents its findings on a dashboard or other standard output. We especially favoured those products that take advantage of the common vulnerabilities and exposures to define vulnerabilities unambiguously.
In general, we found this batch of tools to be an improvement over last year's, and we were impressed with their utility, ease of use and comprehensive reporting.