Product Group Tests

AVA (2008)

Group Summary

Cenzic Hailstorm Enterprise ARC 5.5 is our Best Buy. It is a true enterprise-class product with impressive options and customisations.

We give Db Protect 2007 from Application Security our Recommended award for its flexible configuration options and good integration with other Db Protect tools.

Scroll To Full Group Summary Below

Click for a side by side comparison of products
Click for a side by side comparison of products

Full Group Summary

Tools that assess the vulnerability of the application infrastructure are gaining momentum thanks to ever increasing threats and compliance demands. Nathan Ouellette takes a closer look.

As reliance on information assets continues to grow exponentially, the protection mechanisms at the application layer have created a void between the bad guys and the stakeholders tasked with protecting the infrastructure. As focus has shifted away from traditional network perimeter attacks, client-side attacks and specialised application-layer vulnerabilities now have some IT staff scrambling to catch up. Regulatory compliance mandates such as PCI-DSS and increased industry awareness have created an elevated level of interest in taking the proper steps to secure external facing applications and critical databases.

The gap between developers and infrastructure stakeholders has created a prime opportunity for vendors. Insecure coding practices and database misconfigurations can introduce vulnerabilities into critical application infrastructures that tend to cost more money to remediate after they are sent to production. What we are seeing now is a maturing of application vulnerability assessment products, and it's no surprise.

Because application-layer vulnerability assessments often require manual testing expertise, complimented with automated tools, the products are evolving with that mindset in terms of specialised and dynamic profiling of applications above and beyond point-and-click signature scanning. Also, enterprise management options are helping the products converge into effective tools for administration, testing and remediation that can be easily integrated into a security program. This is evident by the number of products that are introducing LDAP integration, role-based access control, centralised web-based dashboards, highly configurable options and reporting mechanisms that include a large amount of compliance templates and best-practice reports.

The product landscape
We examined three general classes of products: web application vulnerability assessment tools, source-code analysis tools and a database security assessment tool. All three have their place in the software development life cycle and can help security stakeholders make intelligent risk-based decisions in their own right.

The tools in this group all performed above and beyond uncovering the traditional SQL injection, XSS and other standard vulnerabilities. Scanning engines are maturing to the point where the convergence of features and integration into the security program become the differentiator.

How we tested
All products were installed on either Windows XP Professional SP2 host machines or Windows 2003 SP2 servers, as per the requirements of each vendor. We featured four popular enterprise-class database back-ends to ensure testing remained varied.

We ran the source-code analysis tools against two separate sets of extensive and vulnerable test code. Our database vulnerability scanner was run against all four of our database instances, and the web application vulnerability tools were run against three sample web applications containing a myriad of popular critical vulnerabilities.

All products were scored on our typical criteria of support, price and documentation. But we also considered ease of administration and configuration, timed performance of the application to cycle through its targets and whether the product offered features such as compliance templates and reports, remediation reports and the ability to provide role-based access control.

All in all, the majority of our products tested well. Pricing considerations should be given to ancillary features that introduce real value beyond catch rates. We found most of the products to be priced as expected, with some in the higher range. We also saw that some product brands are still in transition following a merger or acquisition. While they remained strong products, support suffers a bit because of the confusion of branding and website content that has yet to make it over to the parent company.

One final note of interest is the recommended screen resolution of the products we tested. Quite often, there is a large amount of information on the screen at any given time and many tasks can be happening in parallel. These are the instances where a high-resolution display really pays off. Getting the information presented cohesively, instead of having to adjust the scrollbar of several different panes at once, can mean the difference between a friendly interface and a cluttered and frustrating one.

- For details on how we test and score products, visit

All Products In This Group Test