Wednesday, October 15, 2014

Home Monitor V.3.0.2 for Splunk Released

After an inspiring .conf 2014, I finally decided to push out the latest version of my Home Monitor app for Splunk, version 3.0.4 maps.

Additionally, I cleaned up many of the dashboards and made them a little more interactive.  They now have time pickers and even an interactive form to track down IP Addresses that are trying to gain unauthorized access to your FiOS router.

New Interactive 'Bad Guys' Dashboard

New Traffic Trends Dashboard

Enhanced Maps Dashboard

New Traffic Flows Dashboard


Lastly, I made some changes on the back end on how the data comes in and gets indexed.  Here are the technical details on what I did, if you're not interested you can stop reading now.

My dilemma was that I'm collecting data from both my pfsense firewall and my parent's FiOS router using a Raspberry Pi syslog server which forwards the data to my Splunk instance at home.  (That can be another post if you're interested on how I pulled that off.)  Since all my data was coming into Splunk as syslog, I needed a way to 'split' the data into two different source types, fios and pfsense.

Here is how I accomplished this task.

First, here is the inputs.conf (copy the sample file in the default directory and move it to the local directory)

inputs.conf

[udp://514]
connection_host = dns
sourcetype = syslog fios
index = homemonitor
disabled = 0

Next, I used the transforms.conf and the props.conf to rename the source type accordingly.

props.conf
[syslog]
TRANSFORMS-changesourcetype = fios, pfsense

transforms.conf
[fios]
REGEX = verizon.net
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::fios
DEST_KEY = MetaData:Sourcetype

[pfsense]
REGEX = firewall.home.com
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::pfsense
DEST_KEY = MetaData:Sourcetype

Let me explain the work flow. First, the inputs.conf file identifies the data coming into port 514 as syslog.  Then, the props.conf file finds the syslog sourcetype and will send it over to the transforms.conf to change the source type from syslog to either pfsense or fios depending on the matching criteria, in my case the REGEX for the host.  Since the fios data comes in from *.verizon.net, I can use verizon.net to trigger the match.  Similarly, my pfsense firewall has a hostname which resolves that I use to match with and convert the sourcetype from syslog to pfsense.
(I'm not ready to push this out to production just yet, so I made the fix in 3.0.4 that fixes the dashboards for all the FiOS users.)

With all that done, now I can switch between both source types and view the charts and graphs for my home network as well as my parent's without having to make any major changes to my configuration files. It's a simple drop down that I have on my dashboards (which I have not included in the v.3.0.4 release but might in a later release depending on demand.)

Thanks and enjoy!

-Kam

6 comments:

  1. Kamilo,
    thanks for releasing version 3.0. It seems to be that there is something missing in the package. After downloading and installing the new release from apps.splunk.com. I receive the following error message when I try to launch homemonitor:

    404 Not Found

    Return to Splunk home page

    Splunk cannot find the "critical_network_traffic_analyzer" view.

    View more information about your request (request ID = 543ec996ce116c93550) in Search

    This page was linked to from http://172.16.0.5:8000/en-US/app/launcher/home.

    ReplyDelete
    Replies
    1. Please look at the latest (3.0.2) release, I had some permissions issues that made it so the /default/data directory did not copy over properly. They have been resolved and thanks for catching that. I attribute this to a lack of sleep related issue.

      Delete
    2. Kamilo, the new release 3.0.2 is working. Thanks for making this app available and solving this.

      Delete
  2. Glad it worked and I hope you enjoy the app! -don't forget to rate it on apps.splunk.com :)

    ReplyDelete
  3. Are there any plans for DD-WRT support in the future?

    ReplyDelete
    Replies
    1. Yes, check out the latest post, I included dd-wrt. I'll create the update shortly and have it available on git and splunkbase.

      Delete