Stephen Singam, managing director, security research, Distil Networks
Stephen Singam, managing director, security research, Distil Networks

In 1905's The Age of Reason, George Santayana wrote, “Those who cannot remember the past are condemned to repeat it.” For us in the IT industry, the emphasis is always on the new and the next, but there are many cases where repetition and remembering our history would help to improve performance. Alongside the monthly update grind caused by Patch Tuesdays, we see the same issues coming up time and again around security.

Theft of user credentials and passwords by hackers will normally get attention from the press, as an unfortunate company falls victim to a hack or software vulnerability. Each one of these attacks will get attention, and many IT professionals will thank their particular deity of choice that it wasn't their systems this time. However, this still looks at each of these events as a single occurrence.

However, we should look at all these thefts over time as part of a major new source of attacks - account hijacking. This covers automated attacks on e-commerce companies and retailers using large sets of password credentials. Each theft of credentials adds to the list of passwords that can be used in these attacks alongside more traditional dictionary attacks or simple substitutions of numbers for vowels.

Every theft therefore represents a new threat to online services that provide shopping or e-commerce purchases. For example, StubHub was hit by a spate of account takeover attempts based on stolen credentials taken from other organisations; the company's IT team was able to make a direct correlation between the theft and the rise in automated brute force login attacks.

This rise in attacks using bots is due to the rise in sophisticated website automation and testing tools. Many of these have valid professional uses - for example, it's easy to test passwords en-masse using tools like Selenium and PhantomJS. For the internal IT team, blocking this vector can make it more difficult for them to maintain their service quality and spot valid issues in the sign-in experience. It's therefore important for companies that test their sites using automated tools such as this to differentiate between “good” tests and “bad” attacks.

Improving security against automated attacks involved looking at activity across accounts in context for patterns driven by bots. For example, a simple approach to stopping automated sign-in attempts would be to limit the number of attempts on an email account or username. However, the most common approach here is to link this to the IP address that the request is coming from. More advanced bots get around this by rotating the IP they present to the site each and every time an attempt is made. Simple IP blocking is therefore no longer effective in dealing with this kind of attack.

Instead, more advanced approaches to spotting patterns can be used to spot similarities in activities being carried out on web pages that can then be used to identify bots. These patterns are normally difficult to detect and involve behavioural analysis specific to the site and how users on the site typically behave. Taking this approach involves understanding how individual human site visitors behave, as well as good bots visiting the site, to understand standard behaviour, and then looking for other activities that are taking place. Some of these may be users behaving in odd ways, while others may be new bot attacks that can then be challenged or stopped.

Alongside this traffic and activity analysis, how sites use APIs and plug-ins should also be considered to harden sites against automated attack. As sites get more interconnected and make use of third party material, this can affect behaviour by users on the site itself. If content on the site is delivered through API calls, then it's possible that those APIs might be attacked as well. Stopping this potential activity should also be considered.

Looking ahead, bots will continue to become more sophisticated in their activities. Already, our research shows that around 80 percent of bot activity displays these characteristics, such as running their own JavaScript to impersonate human browser activity. These bots will take on all the ammunition that they can get, and password breaches happen often enough that fresh material enters the bot landscape all the time. To defend against this kind of attack, we have to learn the lessons of history, so that we can stop needless repetition from causing present day pain.

Contributed by Stephen Singam, managing director, security research, Distil Networks