Monday, September 10, 2012

Home Monitor for Splunk v.2.2

I've just released my first Splunk Application called Home Monitor.  It basically runs on core Splunk and gives you some nice dashbaords, reports and views about the traffic going through your home network.  The application relies on syslog data that is being tracked by your home router.  This data is then sent into your Splunk instance and you can then see what's happening in your network.

Here are the step by step instructions on how to setup your Verizon FiOS router and Splunk. Here's the link http://amiracle19.blogspot.com/p/setting-up-verizon-fios-router-for-home.html .

This latest release has some new pages and dashboards designed to allow you better insight into your home router configurations.  The page "Router Configuration" lets you know if any changes have occurred in the last 30 and 7 days along with a real time check to alert you to change currently happening.

The duration and traffic trends pages give you an insight into what kind of traffic is being brought into your network and which clients are being too chatty.  The Duration page gives you break down by exploitable ports which you might want to guard against.  Lastly, the same duration page lets you know if any of your machines have been compromised since most desktops should not be connecting for longer than a few seconds or minutes to outside sources. 


I have not tested this on the new Verizon FiOS "N" Router but I plan on trying to get access to one soon.

Here is the link to the youtube video showing you how to install and use the app : http://youtu.be/pgJ4dtIn5wo  .  Let me know if you have any questions about the video.

 
Enjoy,
Kam

26 comments:

  1. Just updated the version from 1.1 to 1.2 and added some enhanced searches for the dashboards. I am working on getting the traffic ports properly extracted so I can have a more accurate display of users trying to log into your home system.

    ReplyDelete
  2. HI Can you assist. I have loaded home monitor and I am trying to use with a draytek vigor 2820 router. When I launch your add on I get the following error message. Error in 'stats' command: The argument 'sparkline' is invalid.

    ReplyDelete
    Replies
    1. Hello have you found an answer?
      I have problem like you
      Thank you!

      Delete
  3. Updated to version 2.1 on github and splunkbase, let me know what you think!

    ReplyDelete
  4. Hello, I have a netgear FVS318N router. While splunk is collecting logs from it, the home monitor dashboards shows only a single data item. To confirm that logs are being collected, I went to home monitor -> raw home monitor search and entered "index=homemonitor *". There are 77K matching events. I have tried restarting the splunk daemon. I am running splunk on a Windows7 machine. Thank you.

    ReplyDelete
  5. The proble, you might have is that the fields are not showing up as the same fields which I used in the dashboard. Simply click on the edit button in the dashboard and then click "view results." You will see the search and the results which I wrote. You will need to match the fields with the ones I used in my searches. I am working on building a technology add on for this app which will cover other routers / firewalls. If you want paste a sample from your logs and the make and model of your router and I'll work on that.

    Thanks for your interest!

    ReplyDelete
    Replies
    1. Thank you. You are correct. Editing the search queries worked. Had to also edit some of the UI views. I look forward to the technology app. Thanks for considering the FVS318N. Below are some sample log entries from it. I have asked Netgear for a document describing the format of the log messages. If I get one, I'll let you know.

      Sun Apr 28 10:02:31 2013(GMT-0800) [FVS318N][Kernel][KERNEL] LAN_WAN[ACCEPT]IN=bdg1 OUT=eth1 SRC=192.168.0.9 DST=98.138.253.109 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=13362 DF PROTO=TCP SPT=58755 DPT=80 WINDOW=0 RES=0x00 ACK URGP=0

      Sun Apr 28 10:02:31 2013(GMT-0800) [FVS318N][Kernel][KERNEL] LOG_PACKET[DROP]IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:00:8e:f2:8b:6f:f0:08:00 SRC=192.168.1.1 DST=192.168.1.255 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=520 DPT=520 LEN=32

      Apr 28 10:12:11 192.168.0.1 Apr 28 10:12:10 FVS318N KERNEL [Kernel] LOG_PACKET[ALLOW]IN= OUT=bdg1 SRC=66.235.139.179 DST=192.168.0.9 LEN=576 TOS=0x00 PREC=0xC0 TTL=64 ID=12825 PROTO=ICMP TYPE=3 CODE=4 [SRC=192.168.0.9 DST=66.235.139.179 LEN=1500 TOS=0x00 PREC=0x00 TTL=128 ID=19352 DF PROTO=TCP SPT=59171 DPT=80 WINDOW=16425 RES=0x00 ACK URGP=0 ] MTU=1454

      Delete
    2. I have a raw log file from an ASUS RT-N66U Dark Knight router that I would like to send to you - how can I do so?

      Delete
    3. Check out my latest post and you'll see I've added some support for the Asus router. Let me know if you have any issues setting it up. Thanks!

      Kam

      Delete
  6. Rick,

    The easiest way is to post a sample on this forum and I'll use it to build the Sourcetype. Thanks for your patience, I will soon have an update with the new source types included.

    Thanks,
    Kam

    ReplyDelete
  7. Here ya go:

    Jul 3 13:10:59 192.168.24.1 Jul 3 13:11:00 kernel: DROP <4>DROPIN=eth0 OUT= MAC=60:a4:4c:f0:17:c8:00:14:f1:e4:50:31:08:00 <1>SRC=216.17.8.218 DST=66.169.231.88 <1>LEN=56 TOS=0x00 PREC=0x00 TTL=49 ID=60750 DF PROTO=TCP <1>SPT=443 DPT=52794 SEQ=1584522953 ACK=3730835974 WINDOW=2227 RES=0x00 ACK PSH URGP=0

    Jul 3 13:10:59 192.168.24.1 Jul 3 13:11:00 kernel: DROP <4>DROPIN=eth0 OUT= MAC=60:a4:4c:f0:17:c8:00:14:f1:e4:50:31:08:00 <1>SRC=216.17.8.218 DST=66.169.231.88 <1>LEN=56 TOS=0x00 PREC=0x00 TTL=49 ID=60749 DF PROTO=TCP <1>SPT=443 DPT=52794 SEQ=1584522937 ACK=3730835974 WINDOW=2227 RES=0x00 ACK PSH URGP=0

    Jul 3 13:10:57 192.168.24.1 Jul 3 13:10:58 kernel: ACCEPT <4>ACCEPT IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:14:f1:e4:50:31:08:00 <1>SRC=10.97.0.1 DST=255.255.255.255 <1>LEN=329 TOS=0x00 PREC=0x00 TTL=255 ID=19187 PROTO=UDP <1>SPT=67 DPT=68 LEN=309

    Jul 3 13:10:49 192.168.24.1 Jul 3 13:10:50 kernel: ACCEPT <4>ACCEPT IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:14:f1:e4:50:31:08:00 <1>SRC=10.97.0.1 DST=255.255.255.255 <1>LEN=329 TOS=0x00 PREC=0x00 TTL=255 ID=19161 PROTO=UDP <1>SPT=67 DPT=68 LEN=309

    ReplyDelete
  8. amiracle,

    Thanks for your work on this app.

    I was unable to get Home Monitor to show any data by default. I just saw where I may have to edit some of the fields/searches as mentioned above.

    But now I have what seems to be a bigger problem. When I'm creating the input in Splunk 5.0.1, "homemonitor" is not in the Index dropdown list? So to troubleshoot I uninstalled (stopped Splunk/deleted homemonitor dir/started Splunk) then installed 2.2.1 but the index isn't there...your thoughts?

    ReplyDelete
  9. Check out the lastest blog post, I have added support for Asus, NetGear and Skyhub routers. Let me know if you are able to successfully deploy the updates. Once I've received validation, I will make them a part of the v.3.0 Release.

    ReplyDelete
  10. Great helping Stuff...very useful...
    This is very informative and knowledgeable.I am very glad to read its article.Thank you for useful information... keep it Up!!!
    More info:- Asus Technical Support

    ReplyDelete
  11. I have read about the ASUS RT-N66U not being able to send syslog to a server in their most recent firmware version. Does anyone have this router and can confirm that it still sends syslog and functions with Home Monitor?

    ReplyDelete
    Replies
    1. Yes, this router can send data via syslog. I've tested it and set it up to work with this App. (Advanced Settings -> Administration -> System Tab -> Remote Log Server)

      Delete
  12. i would like to change the port from 514 to 5160

    ReplyDelete
    Replies
    1. Jeremy,

      You can either make the change via the web GUI under Settings -> Data Inputs -> UDP -> 514. Or you can make the change through the configuration file (inputs.conf) Copy the file from $SPLUNK_HOME/etc/apps/homemonitor/default/inputs.conf to $SPLUNK_HOMEetc/apps/homemonitor/local/inputs.conf, then change the port from 514 to 5160. I hope that helps.

      Delete
    2. This comment has been removed by the author.

      Delete
  13. how about adding support for the Astaro/Sophos UTM product line. they have free home versions and paid enterprise versions as well.

    log format is here
    https://www.sophos.com/en-us/support/knowledgebase/115634.aspx

    and Ill test anything you can come up with :)

    Thanks

    ReplyDelete
    Replies
    1. First start by downloading the Sophos TA from splunkbase: https://splunkbase.splunk.com/app/1854/

      Install the TA on your Splunk server running home | monitor > .

      Once installed, edit the props.conf in the $SPLUNK_HOME/etc/apps/Splunk_TA_sophos/default and find the [sophos:firewall] stanza. Modify the src field-alias to src_ip and the dest alias to dest_ip. Create a local directory and save the newly saved file there.

      Open up the transforms.conf file from your home | monitor > app ($SPLUNK_HOME/etc/apps/homemonitor/default/transforms.conf) and copy it to your $SPLUNK_HOME/etc/apps/homemonitor/local directory. Open up the file and add the following entry:

      [sophos:firewall]
      # Make sure that this matches the hostname of your router, pfsense is just an example.
      REGEX = sophos
      SOURCE_KEY = MetaData:Host
      FORMAT = sourcetype::sophos:firewall
      DEST_KEY = MetaData:Sourcetype

      Once you've added this entry, restart your splunk instance and the data should show up in your home | monitor > app.

      Let me know how that works for you.

      Thanks,
      Kam

      Delete
  14. Hi Kam,
    I am new to the world of splunk. I am trying to play with a free version and just got the home monitor app. However I am unable to integrate my modem router which is from netcomm. I went to modem setting and selected "remote"in syslog settings but I do not know which ip address to put for Splunk. Can you pls help?
    Cheers

    ReplyDelete
  15. Vish,

    I think you asked a question on answers about sending data from your modem to Splunk. I answered it there (http://answers.splunk.com/answers/310324/home-monitor-how-to-configure-a-netcomm-modem-rout.html). Let me know if that helps or if you need some additional information.

    ReplyDelete
  16. Hi Kam,

    Giving your app a try, but it isn't going so well for me. Looking at the Home Network Overview screen, the only section that gets populated is the Total Events section. I know the Overview on the SplunkBase page notes that this has been tested with pfSense 2.2.1+. Has anyone else used this with pfSense 2.3.x?

    ReplyDelete
  17. I get a 404 error page not found trying to configure the app. Any thoughts?

    ReplyDelete
    Replies
    1. Yes this is a bug that I covered. You can follow the steps on the wiki to help you fix it: https://github.com/amiracle/homemonitor/wiki/Issues-with-Setup-Page-%28404-Error%29-Fix---work-around

      Delete