The new skeleton key: changing the locks in your network environment

Opinion by Tatu Ylönen

While attending school in Helsinki, I discovered a password 'sniffer' attack in our university network.

While attending school in Helsinki, I discovered a password ‘sniffer' attack in our university network.

To shield our data, I wrote a program to protect information as it moved from point to point throughout the network. I called it the 'secure shell', or SSH for short.

Today, nearly every major network environment – including those in governments, large enterprises and financial institutions – uses a version of SSH to protect data in transit and let administrators manage systems remotely.

SSH works by creating an encryption key pair – one key for the user's machine, and the other key for the server – and encrypting the data that is transmitted between them. Organisations use SSH to encrypt everything from logins to financial data, health records and other personally identifiable information.

While SSH keys protect highly sensitive information, organisations have been remarkably lax at managing SSH key creation, location and access within their network environments. It's as though a home-security company created dozens of copies of someone's house keys, periodically tossed them around the neighbourhood and never bothered to change the lock.

The only factors standing in the way of picking up one of these keys and using it to access encrypted data are curiosity, time and a little know-how.

When organisations are unable to control who creates keys, how many are created, or where they are located in the network after being deployed, they are leaving themselves open to security breaches and non-compliance with regulations.

Behind closed doors

The problem has remained cloaked in the IT department, obscured by its highly technical nature and everyday organisational challenges. System administrators typically see only a small corner of their environments, and may not appreciate or understand the full scope of the issue.

At the other end of the organisation, even if executives and other business managers recognise that there is a problem, they are often simply too busy to investigate its scope or possible consequences.

Yet SSH key mismanagement is as widespread as it is mysterious. Through discussions with major enterprises, governments and financial institutions, we have discovered that, on average, organisations have between eight and more than 100 SSH keys in their environments that permit access to each Unix/Linux server.

Some of these keys also provide high-level root access, leaving servers vulnerable to ‘high-risk' insiders. These insiders, including anyone who has ever been given server access, can use these improperly managed SSH keys to secure permanent entrances to production servers.

Mismanaged SSH keys – a direct path for viruses

The probability of such a breach taking place is increasing by the day. News stories about network breaches are commonplace as attacks become more prevalent and sophisticated. Implementing SSH keys as an attack vector in a virus is very simple, requiring only a few hundred lines of code. Once a virus gains successful entry, it can use improperly managed SSH keys to spread from server to server throughout the organisation.

In fact, key-based access networks are so tightly woven that it is highly likely that a successful attack will infect virtually all organisational servers, particularly if the virus also uses other attack vectors to elevate privileges to ‘root' after breaching a server.

With so many keys being distributed, odds are that the virus will corrupt nearly all servers in a matter of seconds to minutes, including disaster-recovery and backup servers that are usually also managed using such keys.

Under the worst circumstances, a virus using numerous attack vectors could quickly spread internet-wide and, merged with destruction technologies, could destroy immense amounts of data. 

Compliance at risk

Organisations without proper SSH key management protocols in place are not only at risk from security breaches; they are also non-compliant with mandatory security regulations and laws. SOX, FISMA, PCI and HIPAA are all industry requirements that demand both control of server access and the ability to terminate that access. Furthermore, organisations may also be flouting internal security policies (in some cases, policies mandated by customers).

The risks described are not a result of any weaknesses or defects in the SSH protocol itself or its most commonly used implementations. Rather, it is a result of faulty guidelines relating to SSH keys, insufficient time and resources to research the issue to develop solutions, lack of understanding of the consequences of the problem and the hesitancy of auditors to flag up issues that they cannot solve.

It is clear that the issue of SSH key mismanagement cannot be overlooked forever. Without auditing, controlling or terminating SSH key-based access to their IT systems and data properly, most enterprises, government agencies and healthcare providers are sitting ducks for an attacker.

Incorporating best practices

Prior to taking steps to solve a problem, it must first be recognised as truly problematic. It may take several IT teams to begin a remediation project, and will require proper support and endorsement within the organisation itself.

The core of the remediation project comprised multiple steps:

  • Automating key setups and key removals; eliminating manual work, human errors, and reducing the number of administrators from several hundred to virtually none.
  • Controlling what commands can be executed using the key and from where each key can be used.
  • Enforcing proper processes for all key setups and other key operations.
  • Monitoring the environment to establish which keys are actually used and removing keys that are no longer in use.
  • Rotating keys, i.e. changing every authorised key (and corresponding identity keys) regularly, so that any compromised (copied) keys cease to work.
  • Unearthing all current trust relationships (who has access to what).

Going forward

Almost all of the Fortune 500, and many major government agencies, continue to operate out of compliance, and are unknowingly facing major security threats from hackers or rogue employees.

To fully address the issue, it will take several years and thousands of properly trained people. CIOs, CISOs and enterprise IT risk management professionals must make it a priority to ensure that SSH user keys are properly managed in their organisations.

While SSH continues to be the gold standard for data in-transit security, the current threat landscape requires organisations to take pivotal steps to improve the management of access to their SSH networks. 

Tatu Ylönen is CEO and founder of SSH Communications Security


Find this article useful?

Get more great articles like this in your inbox every lunchtime

Upcoming event