• Home
• Process
• Services
• Contact
• Developers

Using splunk to analyse syslog from watchguard

At the company I work for we use Watchguard firewalls to protect our network. I am assured they are 'military grade' and so far so good. We had a Firebox III 500. It blocked everything and did a perfect job of letting nothing in that could harm our network.

The negative about that version was the reporting available from the WSEP (Watchguard System Events Processor). The results were non-readable by non-techies (i.e. my boss) and the version we used stored log files in idx format, also non-readable by third party analysers.

We have since upgraded (read: replaced with a more economic model) the unit with a newer Watchguard unit. This one allows me to save logs in syslog format, as well as the closed watchguard format. I found this pretty exciting as I knew there were a few options around for analysing syslog's, especially as the format is originally from the Linux world.

To do this I needed a syslog server (daemon) which would listen on port 514 for the messages. After a bit of research the best one I found was by Kiwi syslog Daemon. It was supremely simple to set-up and even does log rolling as well as compression, all on a schedule. Pretty good.

With Kiwi set-up and running, next was an analyser. I have a web background so I know about awstats, analog, webalyser, webtrends and Google. So after a few searches fwanalog (based on analog). It takes the syslog, converts it into a 'fake' web server log file, then uses a modified config script to parse the log. AWstats can do something similar but I never tried it. Neither did I demo fwanalog.

What I did know about was splunk. An unfortunate name for something that promises to be quite good. It has funky graphing and has an easy look and feel. It will eat a multitude of log formats (including my syslogs - in zip format) and allow you to search over the logs for almost anything. The possibilities this lends are endless and I am impressed by its scope. There is only one thing that lets it down. The results are only as good as the searches, which means you have to know what you are looking for. As Splunk will read anything thrown at it this could be quite time consuming to those not used to searching through log files, i.e. me.

So I am left thinking there must be a better way to get friendly reports off my Firebox. There is Capra (on Sourceforge) which can parse Firebox logs in native XML format but not syslog. I did manage to get it to run by exporting the old Firebox III logs to syslog format using Watchguard's export tool but that option is no longer available to me. The reports were good if you are a nervous employer.

I am nearing the point of making my own Watchguard syslog analyser, probably based on PHP/MySQL and preset with 'nervous employee' reporting as standard. Check back soon to see how I get on!