Pen testing: How to ensure effective testing

Feature by Rob Buckley

Finding potential weaknesses in your systems before someone else does can save you a lot of trouble. Just follow Rob Buckley's top tips.

Finding potential weaknesses in your systems before someone else does can save you a lot of trouble. Just follow Rob Buckley's top tips.

What do all companies that have ever hit the headlines over an IT security breach have in common? They all thought they were safe. Yet these break-ins happened. Penetration, or "pen", testing is designed to see just how closely self-image and reality match by trying to break an organisation's security.

In IT circles, this usually means attempting to hack into network security and systems, but some organisations also employ physical and social engineering penetration testers to see how easy it is to get people to give away passwords or let unchecked workmen into vulnerable server rooms.

1. Know what you want to defend against

How a pen test is conducted depends on the client, the circumstances, the organisation and the sector in which it operates. Before hiring someone, you need to decide what types of threat you want to defend against.

You may just want to know how resistant to attacks a new system is or whether its installation has upset security with regards to other elements and invite the pen tester to probe the new infrastructure and interconnected systems. This can be relatively cheap and straightforward.

2. Black, white or grey?

You might want to see how a random internet marauder would fare against your defences, and what they could find out about your network. For this, potentially more expensive, "black-box" test, you give the pen tester a URL or range of IP addresses and a time limit and see what they can do.

Or you may be more worried about a current or ex-employee who knows your network intimately. With a "white-box"-style probe, you would bring the pen tester in-house and tell them everything about the infrastructure, maybe even giving them passwords to see what problems might exist if an employee were to try to gain access to systems for which they are not authorised.

"Grey box" tests mix these two approaches. "A commissioning company has to have a clear idea of what it wants," says Paul Vlissidis, technical director of NCC Group. "If they say: 'Can you just test our internal network?' and they've got 2,000 servers, it will take forever if they want a thorough test. But if you know the servers are all the same build, you can just do a sample of ten, for example, and they can learn how to make the build more secure."

3. It's all down to infrastructure

Basic pen tests will usually try to expose vulnerabilities in systems caused by poor patching, bad password choices, open ports, incorrect configuration and other common known vulnerabilities. More advanced or focused tests might home in on particular areas, such as specific applications, custom programs, code testing or SQL injection to try to extract database contents. The infrastructure in your company will determine what kinds of tests might be needed. If your organisation is old and large, you might still have open phone lines that could be vulnerable to war dialling, while wireless networks (whether authorised or not) might be susceptible to war driving.

Even the most innocuous of systems can have vulnerabilities. "There was one company we tested where the servers were well locked down and patched," recalls Shaun Bligh-Wall, technical architect at Vistorm. "But there was a flaw in their backup software that could be exploited to give access to the full system. The administrators hadn't even realised there was an issue."

4. Is it for you?

Choosing when or whether to bring in a pen-testing specialist is typically decided by risk or change management policies. Regulations, due diligence, the size of company, the number of web-facing systems, the organisation's dependency on those systems for its business, the budget available, the cost of a potential breach and other factors will all influence the decision.

"It very much depends on the client and their requirements whether we recommend pen testing," says Lee Newcombe, a consultant at CapGemini and a former pen tester. More often he will recommend a vulnerability assessment that covers a wider range of issues, if not in as much detail.

Determining whether to hire a pen tester and then getting the board to sign off the expenditure is usually a matter of risk analysis, performed either by a dedicated department or by the information security or IT manager. If there's a substantial risk and a breach would incur a considerable cost to the company, your board is more likely to support pen testing. If the senior executives are unreceptive, it may even be possible to bring in a pen tester to explain the potential issues to them.

"It's like paying for insurance," says Ron Meyran, senior product manager at Radware. "Unless something happens, they won't understand why they're paying for it."

5. Don't forget compliance

If you process or store credit card information, you will need to abide by the payment card industry's (PCI) data security requirements, which usually require independent verification of compliance at regular intervals, determined by your company's PCI classification. Most financial services organisations are bound by similar regulations and, because of their higher profile, tend to conduct their own penetration tests as well as use the services of third-party testers to check their systems.

However, small companies that don't host their own systems will have few reasons to spend the thousands of pounds required by a pen tester.

6. Choosing the right partner

Picking a pen tester is far from easy, as hiring someone to break into your network obviously means opening a can of worms. Being able to trust the third party is paramount; knowing it has the skilled people to find vulnerabilities is another critical concern. You don't want to end up paying a large fee to someone for simply running some open-source vulnerability scanners they have downloaded off the internet.

You should also get references from the company, find out how long it has been trading and obtain their consultants' CVs. Use word of mouth: look on forums, check with your industry contracts or any consultants you use and take advantage of any other mechanisms you might have for checking the trustworthiness of a firm you are thinking of using.

Ask testers about their methodology: at the very least, they should be aware of and exceed the Open Source Security Testing Methodology Manual (OSSTMM) and Open Web Application Security Project (OWASP) guidelines. Since these are open source, you can download them from various sources and use them to check the work of any pen tester you hire or follow them as guidelines if you want to undertake any pen testing yourself.

Nick Fisher, a consultant at Unisys, also recommends checking with the Government's computer services group, not just for security information, but for advice about what should be in any contract you draw up with the pen testing company. This will help ensure you get the most out of the deal and are backed up legally in case things go wrong.

7. The benefits of accreditation

Looking for some sort of accreditation is one way of establishing trust. At a basic level, any pen tester who wants to provide verification services to companies trying to prove PCI compliance will need to have been certified themselves. However, the PCI regulations are by no means a comprehensive description of full infrastructure security, and the accreditation scheme is little more than proof that the firm is capable of doing the job, not that it is good at it. For a fuller pen test, other certifications will be necessary.

The CESG IT Health Check (CHECK) accreditation scheme created by the Government for pen testers in the public sector is one such assurance. It comes in levels ranging from red for partial to green for full accreditation; although there will be little difference between the levels for most organisations in terms of the pen tester's capabilities. Although CHECK has almost industry-wide backing, the Government has been redesigning the testing scheme for several months, meaning that newer companies have been unable to get accredited.

8. Presenting the results

It is almost guaranteed that if you have an infrastructure of a certain size and are worried enough to hire a pen tester, then the tester will find something - and no two individuals will come up with the exact same list of issues. What they find and how they present it to you is another way of assessing testing companies. You should always ask for a sample report from the pen tester in advance of using their services. The best ones will not only highlight technical problems, they will categorise by risk, letting you know if there are known exploits or if the vulnerability is purely theoretical, and tell you what you can do about it. The top-flight may provide an executive summary as well as a technical report, depending on the intended audience. The worst pen testers will simply produce a voluminous report that will sit on a shelf gathering dust because there's apparently so much to do that it's impossible to know where to start.

9. Act on the test findings

It's up to you what you do with the information from the report, but plan to allocate some time and money to fixing whatever the pen tester finds. At the very least, you should have some kind of risk analysis strategy to decide what to do about the results of the process. Checking with various sites such as the SANS Institute's for up-to-date exploit information should help you find out whether threats are currently serious, and the advice of other security consultants can also be useful.

10. Put it in your diary

How often you pen test is a decision you need to make for yourself. Most consultants agree that an annual check-up is the minimum for most companies, with additional run-throughs whenever any big changes are made to the infrastructure. You can reduce the frequency to some extent by doing some testing yourself, using methodologies and tools downloaded from the web. But you need time to do this, and it is unlikely you will have the skills to find more complicated exploits yourself. Nevertheless, it is worth doing simply to get rid of the most egregious weaknesses before the pen testers arrive.

Pen testing, when done correctly, can give you assurance that you are reasonably secure in practice as well as in theory. While it cannot guarantee impregnability, it can reveal Achilles heels you never knew you had - before anyone else finds out about them.


Although pen testing has its proponents, many argue that it has flaws. "It is obviously a great way to verify if systems have been built right, as long as you are better than the bad guy," says Roger Thornton, founder and CEO of Fortify (pictured, above). "Where it falls down is the pretty dangerous assumption that the pen tester has as much time as a bad guy." Even if a pen tester comes in for a week every few months, so the argument goes, that's still less time than a determined hacker will have for evaluating every possible chink in your network's armour.

Thornton also highlights the issue of skills. Does someone who has never broken the law have the same insights and abilities as someone who has crossed that ethical line? "People who do the best pen testing are the best hackers," he says. "But do you want to hire a reformed criminal as a tester?"

But if you don't hire the best, are you really testing your network or are you leaving the determined, skilled criminal with an opening?.

Richard Hibbert, chief executive of SureCloud, argues that the differences between pen testers make it hard to compare companies. "For good-quality pen testing, quotes vary wildly from hundreds of pounds to thousands," he says. "And like MOTs, the moment they're done, they're out of date." With no common format for reports, it's hard to see what the differences or similarities might be, even when the companies have been testing the same networks, making it relatively easy to blind people.


The CPP Group provides what it calls "life assistance" to its customers. As part of its services, it keeps its customers' credit-card details and other important facts on file so that if their wallet or purse is lost or stolen, CPP can inform their banks, credit card companies, mobile phone suppliers etc.

Needless to say, CPP needs to be sure its systems are secure to assure clients, prevent any breaches that could reveal confidential information and abide by the terms and conditions of the payment card industry.

"Pen testing is like a safety net for us," says Wayne Armstrong, head of information security at CPP. "It means we have another mechanism for identifying weaknesses that weren't identified during our change management process."

The company performs annual pen tests, having decided as the result of a BS7799 certification project's gap analysis that it needed to find potential vulnerabilities. "The annual pen test we do is over and above any pen test we might do off the back of a project," Armstrong explains. "If we think there's a risk of anything on our infrastructure being exposed as a result of a change we may pen test it as well." Pen testing expenditure is built into both the annual budget and each project's budget.

CPP now uses ProCheckUp for pen testing. "I like the way they work. They use different technology," Armstrong says. He believes that the company's use of an automated, artificial intelligence tool to perform the majority of tests provides an alternative view of vulnerabilities. Certainly, ProCheckUp's tool picked up a serious flaw in a legacy application - grave enough for the application to be turned off until the flaw was fixed - which previous pen testers had missed.

Once Armstrong receives a report from the pen tester, he has to decide which problems are important enough to spend time and money fixing and which are unlikely to cause problems. With most of CPP's systems created by an internal development team, pen testing also provides an insight into where security flaws can arise.

Armstrong plans to continue using ProCheckUp for the foreseeable future, although he doesn't rule out a change at some point in the future. His advice to others? "If you're going to hire a pen tester, use somebody with a reputation you can rely on."


Find this article useful?

Get more great articles like this in your inbox every lunchtime

Upcoming Events