Researchers turn Sun Solaris utility into fuzzing tool

Security researchers have developed a way to turn a utility for Sun Microsystems' Solaris operating system into a rootkit-like, reverse engineering tool that can be deployed to quickly locate application vulnerabilities and create exploits.

The utility, DTrace, is a dynamic tracing, or event logging, function within the Solaris OS that allows systems administrators to monitor a combination of functions, including system performance, statistic debugging information and execution analysis.

Sun designed DTrace to provide operational insights that allow systems administrators to tune and troubleshoot applications and the operating system itself.

Sun released DTrace in 2003 in conjunction with Solaris 10. In 2005, Sun made it available under the Common Development and Distribution License (CDDL) open source license. Apple has since integrated it into its Mac OS X Leopard platform.

The security researchers, Tiller Beauchamp and David Weston, who work at engineering firm Science Applications International Corp. (SAIC), unveiled their findings at the recent Black Hat conference in Washington D.C.

They explained that DTrace provides a framework for performance observability and debugging in real time. With DTrace, system administrators can set probes within their operating environment, then define a metric they want to measure or record.

The tool's ability to take an in-depth look at the operating system and its applications make it ideal not only for reverse engineering, but also for building exploits, the researchers said at Black Hat. Watson called DTrace a friendly programming rootkit that lets you see everything within the operating environment.

One of the key functions of DTrace is its ability to allow automating tasks that would otherwise be manually intensive, Beauchamp said. "If you're sending input to an application to trigger a vulnerability, you can have DTrace alert you when input has reached a vulnerable function. It basically takes a large amount of time off inspecting a vulnerability because it can be programmatically controlled."

DTrace is a great platform as is, Weston added. "But we were interested primarily in a reverse-engineering tool, and DTrace is missing the ability to set conditions that would allow reverse-engineering an application to discover vulnerabilities."

While DTrace is not destructive by itself, combined with other utilities it can cause damage. For instance, it can be manipulated to perform "snooping" operations, such as stealing a user's keystrokes without their knowledge, exactly like a keystroke logger, the researchers said at Black Hat.

Beauchamp and Weston said they have developed a DTrace-based toolkit called RE:Trace. Working with Sun's Chris Andrews, they created a library of routines in a language called Ruby, they told SCMagazineUS.com. With Ruby, they were able to give DTrace a number of capabilities it lacked, including object-oriented programming and expressionals.

Beauchamp and Weston called RE:Trace a "high-level" application programming interface (API) that includes sample scripts. These help not only debug vulnerabilities within applications, but write exploits for them as well, Beauchamp and Weston said.

Tags: Vulnerabilities & Flaws Rootkits

Sign up to our newsletters