A collection of open source software
October 01, 2003
- Ease of Use:
- Value for Money:
- Overall Rating:
Most flexible option.
Steep learning curve. Many tools are command-line only.
Although not for the faint-hearted, no serious forensic or incident response team should be without these tools.
As an alternative to costly proprietary tools, we sourced a variety of open source software, with the goal of emulating as much of the feature-sets offered by closed source products, such as EnCase and ProDiscover.
There is an energetic community for forensics open source, drawing from the resources of forensics professionals, security experts and traditional Unix skills. The result is a toolbox of disparate software products, which, with a little DIY assembly, can achieve results comparable to proprietary products in scope, with the implicit flexibility and extensiveness that only open source can offer.
Not only are the tools available from the community, but several forensics-focused sites exist offering information, advice and even competitions.
To start with, Unix systems usually include, by default, a number of grass-roots tools - essential for forensics work. The most fundamental is dd, which creates bit-for-bit copies of block devices, including disk drives and removable storage. There are also utilities for creating hash sets of files (md5 and sha1), comparing files (comp), searching for content (grep), and identifying specific kinds of files from signatures (file).
We tested products on an Intel system running a base image of Red Hat Linux 9, which, like any distribution, ships with a comprehensive suite of these basic tools.
We then moved on to forensics-specific open source products. First up was Sleuthkit, the forensics project formerly known as TASK, the @Stake Tool Kit. Security specialist @Stake makes much of its software available in open source form, and until recently did so with TASK, before the project was amicably moved to an independent group and renamed.
The Sleuthkit source code is barely 500Kb and required no additional libraries to compile, which it did quickly and easily. Once built, the suite comprises about a dozen tools, which cover a wide range of disk- and file-related functions. For example, dls can extract unallocated clusters, icat copies files specified by inodes, and mactime creates a timeline of file activity. The tools are powerful and precise.
By now, we were able to image disks, mount them in a read-only state, create hash sets and perform basic analysis with a minimum of effort. More difficult was performing detailed analysis of files, searching for evidence and extracting specific files and disk clusters.
As with so many topics, familiarity with the command line makes a world of difference, and getting to grips with the multitude of commands, options and switches is confusing at first. Notably missing is a GUI, partly for accessibility, but also to pull together the repeated menial tasks.
Fortunately, there is just a GUI waiting in the wings: Autopsy, which provides a browser interface to Sleuthkit's tools. A mere 300kb download, Autopsy is a collection of perl scripts, and so requires no compiling, just a working perl environment. Once running, remote or local users can point a browser to the host system. The connection is not encrypted, so for stronger security a VPN would be a good idea.
From within the graphical Autopsy environment, the investigator can create cases and specify which investigators will work on it. Within each case, hosts (sources of evidence) are created, and then images are added to the hosts.
With that groundwork completed, the initial analysis is much easier than from the command line, although it was obvious that some flexibility was lost, highlighting the drawbacks of GUI-only tools. With a couple of clicks, we mounted target images, built hash sets and created file-access timelines.
Autopsy does a good job of pulling together much of the basic functions, such as regular-expression searches, extracting files and data, and identifying file types. In addition to the timeline, investigators can add annotations, allowing notes relating to IDS logs or similar events to be correlated with file activity.
Dropping back to the command line for a while, we tested Foremost, a tool which can extract and identify files from acquired disk images. Foremost uses signatures of known file-types - several hundred - and produces detailed output, with the option to search only at the start of disk sectors, to avoid confusion with nested files (such as within compressed files or email attachments).
With Autopsy able to offer remote browser-based access to forensics investigators, it is fitting that a tool exists to conduct acquisition of remote disks. rda is just such a tool, able to image a disk and transmit the result over the network to a receiving station.
The system must be specifically started from a rda boot-disk (floppy or CD), so this is not an alternative to the live imaging capabilities of EnCase or ProDiscover, but a useful way to avoid physically extracting hard drives. However, the network overhead is intense and so probably of limited use. rda can perform some clever tricks, like spanning acquired data onto multiple volumes (such as CDs or DVDs) at the receiving end.
Having tested only a subset of the open source tools on the market, our inevitable conclusion is that this is a set of tools which any serious investigator must have at their disposal, with the skills to match.
There are times when you may want the comfort of a GUI tool, but the flexibility and power of these tools is undeniable. And, of course, the price is unbeatable.
However, you are on your own for professional training and support, which is why these are better suited as companions to tools with these services available, unless you have highly trained staff with plenty of Unix skills.
SC Webcasts UK
Information Security Manager
Infosec People - Hammersmith, West London
Security Architect, Cardiff - to £70k Basic
Infosec People - Cardiff, Wales
Interim CISO (Chief Information Security Officer) - Cyber Security Director
CYBER EXECS - London (Central), London (Greater)
Junior Penetration Tester, Hertfordshire, to £35k + benefits
Infosec People - England, Hertfordshire
Cyber Security Architect
CYBER EXECS - London (Greater)
Sign up to our newsletters
SC Magazine UK Articles
- Tesco Bank allegedly ignored warnings of hack from Visa
- Investigatory Powers and Digital Economy Bills could threaten economy
- Updated: A million German routers knocked offline by failed Mirai botnet attack
- Gooligan ad fraud malware infects 1.3M Android users, installs over 2M unwanted apps
- Microsoft update left Azure Linux virtual machines open to hacking
- SC Awards Europe 2016 winners announcements!
- ISIS radicalises 'lone wolves' through strong social media presence
- Updated: How will Brexit affect the cyber-security industry in UK and Europe?
- 9.2 million medical records for sale on darkweb
- Microsoft Office 365 hit with massive Cerber ransomware attack, report