Product Group Tests

Risk management (2010)

Group Summary

Skybox 4000 v1.0 is this month's Best Buy for its complete, easy-to-use view of risk.

For its ease of use, we rate Rsam v7.0 Recommended. It has all of the tools required to develop and manage a risk and compliance program.

Scroll To Full Group Summary Below

Click for a side by side comparison of products
Click for a side by side comparison of products

Full Group Summary

Managing risk can be challenging for most organisations. Michael Lipinski reviews nine products that can help.

Risk is defined by the International Organization for Standardization in ISO 31000 as the effect of uncertainty on objectives. This can be positive or negative and we will call these effects impact.

Risk management is the identification and assessment of the elements of risk, those being threats and vulnerabilities. The prioritisation and remediation of these elements is to prevent or mitigate impact. Risk management needs to take massive amounts of information from the entire organisation, correlate to industry regulatory requirements, identify areas where threats and vulnerabilities have the possibility of coming together to provide an impact and provide a usable measure of the value of that risk to the business.

Risk management is a challenge for most organisations. Periodic review of configurations, vulnerabilities, patches, server, user, network and security rules is a challenge. Enterprises operate in a constant state of change and even well staffed and diligent IT staff will be challenged to validate every configuration to corporate policy, test and deploy all necessary patches in a timely fashion and validate end-to-end accuracy of all the security controls deployed.

Risk management is an enterprise initiative and not limited to information technology. Business risk exists where adherence to corporate policies and regulatory requirements is not maintained. Operational risk exists when controls are not deployed to support business policies or regulatory requirements or when those deployed controls are either not effective or can be circumvented.

For this review we looked at products that could measure, analyse and report risk within an enterprise. We looked for the products to report within the formats and frameworks of multiple regulatory requirements (e.g. SOX, GLBA, PCI DSS, ISO, etc). We also looked for solutions that were network-centric, could centrally collect and store data, had centralised analysis and reporting, were centrally managed and were focused exclusively for risk management.

Additional functionality we were looking for included the ability to: collect data across the network, including threats and vulnerabilities; report associated risk; provide remediation options (beyond what traditional patch management systems deliver); and report based upon regulatory requirements and local policies.

Our testing methodology for this review utilised vendor-provided web reviews. Suppliers were allowed to run through a short presentation on the company, product features and value proposition and describe the implementation process a typical end-user would experience. We then ran through a full demonstration of the products using our usual evaluation criteria. We asked the participants to not only demonstrate the features and capabilities of the offering but to also run through a typical deployment scenario.

The solutions reviewed consisted of client-side software deployments (usually server software and agents), appliance-based solutions and hosted SaaS offerings.

The products focused on the business side of risk, the creation of policy and adherence to those policies and to multiple regulatory policies. Some were operationally focused and collected information from the various deployed security controls and network systems to validate those policies against corporate policy and industry regulatory compliance. A number of the solutions collected vulnerability information from industry standard scanning tools and correlated that vulnerability data to the operational threat data it analysed.

Just as risk has become an overused term today and used for many security, policy management and vulnerability analysis tools, these solutions looked to address risk in very different ways. Regardless if they focused on the business policy and adherence side or the operational controls side, we focused our review on the ability of the products to identify and analyse risk, measure and report on the risk, easily compare to industry compliance regulations and provide remediation options in an easy-to-use interface.

We were very impressed with all of the products in this review, as the maturity level really showed. The ease of use and the user interfaces have been designed to present very large amounts of data in easy to follow formats. Reporting and alerting were very strong. All of the solutions came with a substantial amount of content, covering all the standard regulatory bodies, extensive reporting templates and numerous sets of questions for developing assessments.

As always, choose carefully as all of these solutions use content for various decision making and analysis. The ability of the vendors to keep that content updated and fresh is very important. Some of the operational focused solutions deployed rapidly. Others required between a couple and several months to fully deploy and implement within the organisation.

All Products In This Group Test