Richard Thomas served as UK information commissioner until 2009. He is credited with bringing data security into the mainstream. By Paul Fisher.

What was the highlight of your time as the information commissioner?
Where to start? Almost every Freedom of Information case was new. All sides were testing the boundaries of a complicated and culture-changing Act and the Information Commissioner's Office (ICO) was on the same steep learning curve as everyone else.

The role of the Act in exposing MPs' expenses ignited public awareness and the consequences still reverberate across the political landscape. It was a strange experience to be told by two MPs I was the most unpopular man in Parliament – and this was before the newspaper leaks.

There were two big Iraq war cases centring on the Attorney General's advice and the Cabinet minutes pre-invasion.

Of course, the loss of 25 million child benefit records at HMRC marked a turning point for data protection.

This has led to stronger powers for my successor, Chris Graham. The ICO can also claim credit for putting concerns about surveillance on to the agenda; we produced a major report in 2006 and hosted an international conference.

During the past six years, data protection and freedom of information have emerged from the shadows. But when I started as information commissioner in 2002, freedom of information was largely a subject for the so-called chattering classes and an unknown and rather threatening prospect for those in the public sector.

I knew that implementation of such a wide-ranging and comprehensive law was going to be a major challenge.

Data protection was seen as remote and complicated. The public often saw it as stopping things happening rather than a safeguard for their private information.

When I stood down in mid-2009, there was plenty of unfinished business, but I am proud at how far we have come. Transparency is now central to the political and media vocabulary and freedom of information is recognised as bringing greater accountability. There can be few now who are unaware of the need to control the risks of data abuse.

But, I guess, for a long time I will probably be remembered for asking the question: “Are we sleep-walking into a Surveillance Society?”

Following the HMRC scandal, can the public have faith that public institutions take data security more seriously?
The loss of the child benefit details was a massive wake-up call. The media and the public were aghast – it brought home how personal data can be very damaging in the wrong hands. The politicians were acutely aware that the loss raised questions of public trust and confidence.

Officials were embarrassed so much damage could flow from non-existent or ignored procedures, inadequate IT safeguards and a great deal of human error. People were unaware of the power of technology. I lost count of how many said to me: “I had no idea you could get so much data on two CDs.”

We had been warning of the risks for some years, but of course the temperature rose higher still as more and more data losses surfaced in the months following the HMRC incident – personnel records on an unencrypted MoD laptop, the entire prison population on a stolen memory stick, bank and life assurance records... the list went on and on. Have things got better? Probably. There is now far better awareness and it has got to the top of most organisations. This is fundamentally a reputation issue for government and businesses. Losing data looks very bad. The PwC report into the HMRC incident and Sir Edmund Burton's report into the MoD laptop loss both contained hard-hitting analysis as to what had gone wrong. The recommendations of these reports were used by my office as the basis for formal enforcement notices, with criminal sanctions, which we served on both HMRC and the MoD.

This was followed by plenty of guidance in the Data Handling Review led by cabinet secretary, Sir Gus O'Donnell, with a series of mandatory requirements for all government departments. Private sector bodies would do well to benchmark themselves against it – especially since the ICO will from April be able to impose substantial fines.

Has the economic crisis made managing data harder?
There may be less money around, but that cannot make a difference to the priority for handling data properly. Modern IT gets ever cheaper and offers massive scope for efficiency saving, but the economic crisis cannot be an excuse for cutting corners. The problems get worse as storage capacity increases exponentially and devices become more mobile. There is also a generational issue. Too many workplaces are now populated by the Facebook generation; it has a casual attitude towards its own information, but is less aware of the need for a disciplined approach at work.

I am glad more organisations recognise the risks of handling large volumes of personal data and approach the challenges in terms of risk assessment.

Above all, there must be someone responsible at the top of the organisation who is able to cut across and co-ordinate these functions. Data protection cannot be left only in the hands of the lawyers, the IT departments or the training departments – they must all be involved.

How can we ensure we both meet our security needs and protect civil liberties?
My office tried to answer this question in the report, A Surveillance Society?, that we published in 2006. There is a tension and the right balance must be achieved. Profiling and database mining can be powerful weapons in the fight against terrorism and serious crime. But there is no silver bullet. Technology alone will not remove the threats and it can bring its own problems. Too much information can swamp systems and give false security or allow the bad guys to get through.

And if the arrangements are too intrusive or burdensome, our traditional freedoms and liberties are put at risk. As commissioner, I argued successfully that a state-run database with records of everyone's communications data (phone calls, emails and internet activity) would be a step too far for the British way of life. The right balance involves hard scrutiny of purposes, proportionality, effectiveness, safeguards and less intrusive ways of achieving the same ends. I am delighted privacy impact assessments, as proposed by my office, are effectively mandatory for all new schemes.

What challenges lie ahead for data and information security professionals?
How's this for starters? Meeting expectations that often conflict with each other. Ensuring that top management understands the power and the risks of what they are installing. Getting them to take information governance more seriously. Communicating in Plain English. Remembering that the ICO now has stronger inspection powers and sanctions. Getting to grips with the reality of international data flows. Looking forward to a more effective European Directive on data protection.

You've joined the Centre for Information Policy Leadership. What is your role there?
The Centre for Information Policy Leadership (CIPL) is the world's leading data privacy and information think-tank. It was founded in Washington DC in 2001 by the top privacy law firm Hunton & Williams. It was established to develop innovative, pragmatic approaches to privacy and information security issues, and the centre has stimulated international discussion of many key privacy issues, including governance models for international data transfer, privacy legislation in emerging economies, transparency and government use of private sector data. Its knowledge bank is publicly available, and is used to inform policymakers and privacy professionals worldwide.

How do you relax?
I have been fortunate to enjoy a number of challenging jobs. But my family has always been priority number one. I have been married to Julia for 36 years and we have three adult children, of whom we are very proud. We enjoy walking and foreign travel.

I also relax by reminding myself that information security professionals are trying their hardest to get things right.