Strengths: Huge range of log sources supported, database indexing provides fast searches, log data can be used for alerting, sophisticated reporting
Weaknesses: The base version is free, so no complaints here
Verdict: With Splunk on your side, log data collecting and indexing just got a whole lot easier and more cost-effective as well
Gathering log data is an essential function of network security but businesses are faced with tough choices as the majority of network monitoring tools can't index this information for fast searches, while point solutions can incur a price premium. Splunk gathers log information from many sources, stores it in a central database and indexes it. If you don't store more than 500MB of data a day, it's free.
You can configure Splunk to listen on IP address/port combinations, and any text log data, such as syslog and log4j, will be retrieved, imported into the database and indexed. Audit daemons such as those on Linux and Solaris systems create log files as binaries. To read these, Splunk needs them converted to text format, which is achieved via utilities provided by the OS vendor, which Splunk can run using its built-in scheduler. It picks up text data as it is created, imports it into its database and indexes it.
Other possibilities are folders and files where you have a local folder on the Splunk server or on a remote system to store text log files. Splunk can monitor selected folders and index data from these log files into its database. As the file grows, it will only take new data from it and index that.
Windows systems can be monitored as Splunk indexes the Event Log, registry and WMI data. The registry for each system is gathered and indexed, and from this point, all registry changes are added as they occur. You need to upgrade to the enterprise version to store more than 500MB of daily log data and other key differences are support for a distributed server environment and user authentication for database and search access. In a distributed environment you would have multiple systems running Splunk, gathering data from different areas of the network and sending it to a central server. Each system can index data before it passes it on, or you can leave the central server to carry out all indexing.
Splunk took a few minutes to install and its web server home page opens with a dashboard showing a list of all log sources, source types and monitored host systems. You can add graphs or tables showing anything from firewall or router log data or security violations. You start by defining log data sources, or inputs - these can be host systems, ports, folder locations or FIFO queues. We wanted to play with syslog sources and configured Splunk to listen on port 514 for all LAN IP addresses so syslog data from any source would be picked up. The crawl option is useful as you can get Splunk to scan systems, volumes or folders and list the files it finds, which can be refined.
We had a RadWare DefensePro DP3020 IPS appliance fronting our network - this was configured to send syslog data to the Splunk server. A port scan on our external IP address to simulate an attack was found. We could easily select syslog data from Splunk's home page.
From the syslog data, we could see the device name, event and security policy that was triggered. With this data, we could configure Splunk to alert us by email.
Due to the number of features and capabilities on offer, Splunk presents a steep learning curve, which isn't helped by the muddled web-based documentation.
Nevertheless, once we got to grips with it we found Splunk capable of collating a vast amount of log data and teaming this up with fast, sophisticated search capabilities, making it an ideal partner for network administrators and security auditors.