Setting up Verizon FiOS Router for Home Monitor **UDPATED**

Log into your router and select Advanced, click Yes to proceed.

Select "System Settings"
Scroll down and enable System Logging and Security Logging.  Next, Enter the IP Address of your Splunk Server.

Now select the Firewall Settings
Click on the Security Log
Click on the Settings button.

Put the check box on all the items you want logged in Splunk, then click Apply.


Now log into your Splunk instance and go to the Manager.


In Splunk, select Add Data and then select Data Inputs.

Click on UDP

Follow each step and MAKE SURE to click on the check box for More settings. When complete, click Save.

When done, your Data inputs page should look like this.

50 comments:

  1. The homemonitor index was not automatically created when installing the app. I had to manually create it.

    ReplyDelete
    Replies
    1. Just fixed it in the latest version 1.2. Thanks and good catch!

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. I'm getting no results found..

    ReplyDelete
  4. On your Splunk server, can you do a tcpdump and look for traffic on udp port 514?

    ReplyDelete
  5. No I'm on Windows 8. I went in and allowed port 514 UDP through the firewall and still no dice.

    ReplyDelete
  6. I would run wireshark on your Splunk instance and validate that you are seeing traffic from your FiOS router. You can also validate if Splunk is receiving data by taking a look at the homemonitor index. Search index=homemonitor * All Time.

    ReplyDelete
  7. I can see traffic... homemonitor index event count shows only 15 from yesterday all with the following:

    Jan 15 22:58:21 192.168.1.1 Jan 15 23:58:20 2013 Wireless_Broadband_Router Unknown PTR name format
    host=192.168.1.1 Options| sourcetype=syslog Options| source=syslog Options

    ReplyDelete
  8. This could mean a couple of things. First, what version router do you have? (Model number and firmware version). Next, what options did you select on your routers configuration page, Information, Warn, or Error?

    ReplyDelete
  9. Router Info:

    Model Name: MI424WR-GEN3I
    Firmware Version: 40.19.36
    Hardware Version: I

    Both System and Security logging are set to information

    ReplyDelete
  10. In your Splunk instance, go to Manager->Data Inputs->UDP Click on 514 and when it opens, make sure that "Set Sourcetype" is either set to "From a List" and syslog is selected, or you can use "Manual" and type in syslog. Let me know if that works.

    ReplyDelete
  11. Hi, I followed your instructions completely, but I cannot get it to work. It looks like my Fios Router is blocking the syslog traffic because all of the dashboards have the message "No results found" I would greatly appreciate your suggestions.

    ReplyDelete
  12. The first thing I would do is just do a search in Splunk like this: index=homemonitor and go ahead and do it for All Time. If you see results, then the data is properly flowing into the index. If not, we need to revisit your setup. Are you running your Splunk server on Windows or Linux (Mac OS X)?

    ReplyDelete
  13. Hi, after doing more research yesterday, I think the issue was that I did not install Splunk as root so it could not listen on ports <1024. I added redirect rule to iptables and mapped UDP 514 to UDP 10514. I think that I did this correctly, but I was winging - I am using Ubuntu 14.04 and could not use the instructions that found on splunk>answers. I also cloned the data input rule for UDP port 514 and created one for UDP port 10514. Index=homemonitor* has a lot of data and the all time & realtime (30 second window) views are updating, but the dashboards still show “No results found.” Thanks for your help.

    ReplyDelete
    Replies
    1. Did you try running tcpdump on your Splunk box to validate that you're receiving traffic from the FiOS router? (tcpdump -i eth0 UDP) Also, is there a reason you're not running Splunk as root on your Ubuntu box?

      Delete
  14. This comment has been removed by the author.

    ReplyDelete
  15. I was mistaken about installing by root (I'm new to Linux). I followed the install video on the Splunk website, and I think it was referring to not switching to the root user. Regardless, I took the following steps:
    1) Uninstalled Splunk (sudo dpkg -P splunk)
    2) Reset iptables (sudo iptables -F)
    3) I executed (sudo tcpdump -1 wlan0 udp) and got the following output:

    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on wlan0, link-type EN10MB (Ethernet), capture size 65535 bytes

    followed by a stream that looks like this:

    ##:##:##.###### IP Router.home.domain > @@@@@@@-wifi.home.38127: 6884* 1/0/0 PTR @@@@@@-wifi.home. (74)
    ##:##:##.###### IP @@@@@@-wifi.home.##### > Router.home.domain: 26323+ PTR? 8.4.b.8.5.c.e.f.f.f.1.c.1.3.a.7.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (90)
    ##:##:##.###### IP FiOS_Router.home.1026 > @@@@@@-wifi.home.syslog: SYSLOG user.info, length: 183
    ##:##:##.###### IP FiOS_Router.home.1025 > @@@@@@-wifi.home.syslog: SYSLOG user.warning, length: 60
    ##:##:##.###### IP Router.home.1026 > @@@@@@-wifi.home.syslog: SYSLOG user.info, length: 181
    ##:##:##.###### IP Router.home.1026 > @@@@@@-wifi.home.syslog: SYSLOG user.info, length: 163
    ##:##:##.###### IP Router.home.domain > @@@@@@-wifi.home.#####: 26323 0/1/0 (149)

    4) Reinstalled Splunk (sudo dpkg -1 splunk-6.2.1-245427-linux-2.6-amd64.deb)
    5) Started Splunk and installed Home Monitor
    6) Configured the data input as follows:
    a) UDP port 514
    b) Source - Source name override (blank)
    c) Set sourcetype * (Manual)
    d) Source type * (syslog)
    e) Set host (custom radio button selected)
    f) Set the host with this value. (blank)
    g) Set the destination index for this source. (homemonitor)
    h) Only accept requests from this host. (blank)

    At this point, I can see a stream of data when I search: index="homemonitor", but the dashboards are still empty.

    Thanks for your help!

    ReplyDelete
    Replies
    1. What does the output look like when you run index=homemonitor in Splunk?

      Delete
    2. 1/19/15
      11:32:23.000 AM
      Jan 19 11:32:23 192.168.2.1 Jan 19 11:32:22 2015 FiOS_Router OUT: ACCEPT LAN-OUTBOUND [15] Default policy (TCP 192.168.2.60:55904->###.###.####.###:443 on eth1)

      host = 192.168.2.1
      source = udp:514
      sourcetype = syslog

      1/19/15
      11:32:23.000 AM
      Jan 19 11:32:23 192.168.2.1 Jan 19 11:32:22 2015 FiOS_Router OUT: ACCEPT [54] Connection opened ( : TCP 192.168.2.60:55904 <-->###.###.####.###:55904 [###.###.####.###:443] CLOSED/CLOSED eth1 NAPT Outgoing UNSECURED )

      host = 192.168.2.1
      source = udp:514
      sourcetype = syslog

      1/19/15
      11:32:23.000 AM
      Jan 19 11:32:23 192.168.2.1 Jan 19 11:32:22 2015 FiOS_Router OUT: ACCEPT LAN-OUTBOUND [38] Wireless Broadband Router initiated traffic (UDP ###.###.####.###:1024->###.###.####.###:53 on eth1)

      host = 192.168.2.1
      source = udp:514
      sourcetype = syslog

      1/19/15
      11:32:23.000 AM
      Jan 19 11:32:23 192.168.2.1 Jan 19 11:32:22 2015 FiOS_Router OUT: ACCEPT [54] Connection opened ( : UDP ###.###.####.###:1024 <-->###.###.####.###:1024 [###.###.####.###:53] eth1 Route Outgoing UNSECURED )

      host = 192.168.2.1
      source = udp:514
      sourcetype = syslog

      1/19/15
      11:32:23.000 AM
      Jan 19 11:32:23 192.168.2.1 Jan 19 11:32:22 2015 FiOS_Router OUT: ACCEPT [54] Connection opened ( : UDP 192.168.2.60:60507 <-->###.###.####.###:60507 [192.168.2.1:53] local_dev Route Outgoing UNSECURED )

      host = 192.168.2.1
      source = udp:514
      sourcetype = syslog










      Delete
    3. Hi, can you tell what I am doing wrong from this excerpt from index=homemonitor or do you need more info? Thanks

      Delete
    4. I figured out what the problem was with the app. It's a simple fix, you just have to go into your Data Inputs (Settings -> Data Inputs -> UDP -> click on the 514). Change the sourcetype from syslog to fios. That will make all your dashboards populate. I've been working on something that will fix that in a later release. Good catch!

      Delete
    5. I updated the app but the dashboards are still empty and now when I search index="homemonitor", there is no new data being added to the index. The output from sudo tcpdump -1 wlan0 udp looks the same.

      Delete
    6. Let's try and edit the panels and see if we can get them to populate with your data. First, let's make sure you're getting the data with the correct sourcetype. Go into search and type: index=homemonitor | stats count by sourcetype

      You can do it for the last 15 minutes.

      You should see the sourcetype fios and no syslog.

      Send me an email, kamilo@gmail.com and we can do a quick webex or Google Hangout.

      Delete
    7. Hi, I started from scratch and reset my router back to its default settings and it fixed the problem. Thanks for your help - the dashboards looks great!

      Delete
  16. This comment has been removed by the author.

    ReplyDelete
  17. I just updated the app to version 3.0.4 that has all the fixes listed above. Please feel free to download it and test it out with your system. Thanks for pointing out the issues that were in the app.

    ReplyDelete
  18. Home monitor and pfsense 2.2.1. The search arguments in home monitor are not compatible with the input from pfsense. I have lots of input from pfsense but the surch arguments in the panels. Give zero hits.

    ReplyDelete
  19. This comment has been removed by the author.

    ReplyDelete
  20. I have a vanilla install of Splunk on a Macbook Pro. I have only installed Stream and Home Monitor 4.0.1. I have reviewed the installation video on youtube for version 2.x and read the blogs. I followed the installation instructions provided. The router was configured to forward syslog to my laptop and I can verified 514 traffic is being received with tcpdump. I created the UDP Data Input, but didn't see FIOS as a source type from the pick list. It's set for syslog. I'm not sure what I'm missing

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. This comment has been removed by the author.

      Delete
  21. I have a vanilla install of Splunk inside a CentOS7-vm on my Macbook Pro, I have fusion8pro installed there where the VM runs. I have followed the directions above to the T and I am not seeing anything in the dash board. I have configured the inputs to account for up\514 and I did select the index. One nuance I did notice was that a UDP connection for syslog was in data inputs. So I had to modify that one for the advanced settings. no clue how to fix, would love to see the data......

    ReplyDelete
  22. I'm working on a video that will walk you through the install of Splunk, the home | monitor > app and get the data into Splunk from your router / modem.

    The easiest way to fix the input is to go into the $SPLUNK_HOME/etc/apps/homemonitor/default and copy the inputs.conf file to the $SPLUNK_HOME/etc/apps/homemonitor/local directory and make the modification there. Once the changes have been made (change the port or whatever change you want to make) then it should start receiving the data.

    Make sure that you've either disabled iptables (sudo service iptables stop) on your Linux server OR just add an entry for UDP 514 in-bound (modify /etc/sysconfig/iptables/). Let me know if that works and if you don't see any data.

    ReplyDelete
  23. Can you add support for D-Link router, mine is DIR-655. thanks

    ReplyDelete
  24. I am familier with Splunk but new to Home Monitor App. First of all your app is amazing, thank you.

    So I got my dd-wrt log to populate and its working great. Then I moved to fios router. I noticed there is no sourcetype where it was mentioned in earlier comments. Do I need to create that sourcetype and if so is the parsing parameters documented somewhere, I couldn't find it?

    Currently using Splunk 6.3.3 and Home Monitor 4.4.2

    Thank you in advance!

    ReplyDelete
    Replies
    1. Yebro,

      Thanks for downloading and using the app. If you want to change back your sourcetype, you can either do this via the Web GUI or modifying a configuration file using the CLI. The WebUI option is good if you're going to stick with your FIOS sourcetype; however if you’re going to be going back and forth between different routers, it might make more sense to make the change to the configuration files.

      The WebUI option is simple, just go into Settings -> Data Inputs -> UDP 514 and then over ride the source type as fios.

      The CLI option is a more scalable solution if you’re going to be jumping around from different sources. First, get the hostname for your router. For this example, I’m going to use “home.fios” as the hostname. To make the change, log into your Splunk server and go to $SPLUNK_HOME/etc/apps/homemonitor/local and edit the transforms.conf file. The transforms.conf file typically does not exist in this directory, just simply create it and add the following entry. Under the REGEX = field, just make sure that you set the entry to your hostname.

      [fios]
      # Make sure that this matches the hostname of your router, fios is just an example.
      REGEX = home.fios
      SOURCE_KEY = MetaData:Host
      FORMAT = sourcetype::fios
      DEST_KEY = MetaData:Sourcetype

      Once you’re done making the changes, go ahead and restart your Splunk instance. Now, let’s say you want to add a dd-wrt router (hostname = ddwrt.home), you can add this line to the same transforms.conf file :

      [dd-wrt]
      # Make sure that this matches the hostname of your router, fios is just an example.
      REGEX = ddwrt.home
      SOURCE_KEY = MetaData:Host
      FORMAT = sourcetype::dd-wrt
      DEST_KEY = MetaData:Sourcetype

      Now, anytime a machine sends data in as syslog then Splunk will look at the hostname. If it matches ddwrt.home, it will set the source type to dd-wrt; or if it’s home.fios then it will set the source type to fios all automatically.

      I know it’s a long winded answer, but that should cover you. Let me know if you have any additional questions.

      Thanks,
      Kam

      Delete
    2. Thank you Kam,

      As for props.conf file, how would the fios parsing parameters look like. I wasn't able to see "fios" as a sourcetype however when I tried creating it, it said I was already created.
      Thank you again

      Delete
    3. If you want to see how the fields are created for the fios sourcetype, go to $SPLUNK_HOME/etc/apps/homemonitor/default/props.conf . Don't modify the file there, any changes will be overwritten when I make any updates.

      Delete
    4. Thank you Kam,

      So I tried using those fields for my fios box but did not match. Here are few examples from my fios box:

      Feb 17 00:22:29 192.168.1.1 Feb 17 00:22:28 2016 fios IN: ACCEPT [57] Connection closed ( : UDP 8.8.8.8:53 <-->8.8.8.8:53 [192.168.1.2:59059] br0 Route Incoming UNSECURED )

      Feb 17 00:22:12 192.168.1.1 Feb 17 00:22:11 2016 fios OUT: ACCEPT LAN-OUTBOUND [38] Wireless Broadband Router initiated traffic (UDP 100.29.130.190:1024->68.254.0.13:53 on eth1)

      Feb 17 00:22:12 192.168.1.1 Feb 17 00:22:11 2016 fios Empty name

      Delete
    5. Got it to work. I know not to tweak this file /opt/splunk/etc/apps/homemonitor/default/props.conf but I noticed no way to get it working with current config. so here are my changes/correction.

      [fios]
      DATETIME_CONFIG =
      NO_BINARY_CHECK = true
      category = Custom
      disabled = false
      pulldown_type = true
      EXTRACT-direction = ^(?:[^ \n]* ){9}(?P[^:]+)
      EXTRACT-protocol = ^[^\(\n]*\(\s+:\s+(?P\w+)
      EXTRACT-action = ^(?:[^ \n]* ){10}(?P\w+)
      EXTRACT-state = ^[^\]\n]*\]\s+(?P\w+\s+\w+)
      EXTRACT-src_ip = ^[^\(\n]*\(\s+:\s+\w+\s+(?P\d+\.\d+\.\d+\.\d+)
      EXTRACT-src_port = ^(?:[^:\n]*:){7}(?P\d+)
      EXTRACT-nat_ip = ^[^>\n]*>(?P\d+\.\d+\.\d+\.\d+)
      EXTRACT-dest_ip = ^(?:[^\[\n]*\[){2}(?P\d+\.\d+\.\d+\.\d+)
      EXTRACT-dest_port = ^(?:[^:\n]*:){9}(?P\d+)
      EXTRACT-interface = ^(?:[^\]\n]*\]){2}\s+(?P[^ ]+)
      EXTRACT-reason = ^[^\[\n]*\[(?P\d+)
      EXTRACT-config_change_user = ^[^\(\n]*\(\w+\s+\w+\s+(?P\w+)
      LOOKUP-fios = action_lookup action OUTPUTNEW action2
      LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host

      Delete
  25. This comment has been removed by the author.

    ReplyDelete
  26. That's great, if you want just copy that stanza to your $SPLUNK_HOME/etc/apps/homemonitor/local/props.conf file. This way when I make an update it will not overwrite your changes.

    ReplyDelete
  27. Hello Kamilo,
    Is google maps no longer available via Splunk Apps or add ons? I'm not longer able to find it through splunk enterprise, the latest version 6.3.3. I was trying to install it as instructed by the video. Any insight?

    thanks in advance!

    ReplyDelete
    Replies
    1. I think I just answered my own question, It's not available in Splunk Enterprise 6.x - so that makes me wonder about home monitor and the latest 6.x enterprise version and mapping....just wondering if you're now using built in splunk maps or some other method? keep up the good work!

      Delete
    2. Yes I started using the defaul maps that come with Splunk enterprise.

      Delete
    3. So when you say default maps that come with Splunk Ent, that means that i do NOT have to download other apps to install correct? I cannot get any of the maps to work unfortunately. My input is fios, when i click on Map of Connections.....just sitting stale on Waiting for input.... is there any trick to getting these maps to populate on that dashboard? any files to manipulate? thanks in advance.

      Delete
  28. Kamilo,
    appologies, reposting as its own post now.
    just wondering if there is any update to Craigs issue, I didn't see a remedy/response in this thread. I've been poking around many related threads and I'm still having issues.

    I have a similar problem, I'm using the latest version of Splunk Enterprise (6.3.3) on a mac mini, El Capitan, with the latest version of Home Monitor (4.4.2).

    I've been reading several blogs and searched this one and the videos. That said, I think I have it correctly configured, please let me know if i'm way off; In Data Input - I've configured the UDP 514 as instructed, but with the update of using 'fios' in the place of 'Source name override' , set source type manual, picked from the list 'syslog', then I checked the more settings, and for host, selected the button for custom. I've tried whatever popped up as default here, left it blank, and since one of your posts says use the router hostname (i take it that that means the fios host name, your router hostname, found in Fios system settings as 'Wireless System Hostname' in which i put as 'fios')
    note from https://4dd0p3r470r.wordpress.com/2015/01/31/how-to-splunk-home-monitor-for-fios-routers/
    I followed this as well as this was a little vague in repeated postings: IMPORTANT UPDATE(!): in the field “Source name override” put “fios” instead of the “syslog” - i did this - let me know if thats wrong too.

    which leaves the final thing in the Data set , i chose homemonitor as the index, and hit save....

    unfortunately, nothing is populating in my homemonitor. Not sure what I'm doing wrong here.

    I also saw this thread; https://answers.splunk.com/answers/310324/home-monitor-how-to-configure-a-netcomm-modem-rout.html#answer-310832

    in which you answer about Mac osx , 'If you're running Splunk on Linux (or Mac OS X), then you'll either have to run Spunk as root or change the port that your modem is sending syslog to 1514 UDP.'
    -I assume I am running as root, im the Admin on the machine, i even enabled the root user, but not sure if thats an issue with simply executing the Splunk icon when i launch it. so I think it would run as me being the admin which should be root - let me know if i'm mistaken.
    'change the port that your modem is sending syslog to 1514 UDP' - I assume that you are saying change your Fios router to send the syslog via port 1514? I cannot figure out how to change my Fios router to change that......(i also tried changing the Data Set back and forth to 514 and 1514 via the web UI and tried hard coding into the .conf files) niether of which made a difference.....still no traffic coming to the splunk server from Fios router.

    I also used wireshark to check for anything UDP coming from my Fios router, and i havent seen a single packet for hours using 514 and 1514, i even cleared my logs and started them over to generate current traffic. My NTP is synched too on fios and splunk.

    I guess that might explain why i cant get home monitor to show anything as well on Data Summary and trying in the search index=homemonitor sourcetype=fios, and index=homemonitor | stats count by sourcetype

    I've burned 8 hours on this today - really not sure what else to try....any ideas? by the way, great work! I really love what it can do and cant wait to get it working! thanks in advance.

    ReplyDelete
    Replies
    1. First, when you run Splunk you are probably not running it as root. In order to run splunk as root you will have to go into your terminal and type sudo "$SPLUNK_HOME/bin/splunk start" . That will start splunk as root on your machine. If you want to have it start automatically as root, you will need to run "sudo $SPLUNK_HOME/bin/splunk enable boot-start -user root"

      I've used the environment variable $SPLUNK_HOME, if it's in your profile, then you should be fine to run the commands as listed above, otherwise go to your splunk home directory (/Applications/Splunk I assume) and run sudo bin/splunk start . It will prompt you for your admin password and then run the command.

      Once you've finished that, then you should start seeing data come into your Mac on port UDP 514.
      (http://docs.splunk.com/Documentation/Splunk/latest/Installation/InstallonMacOS)

      Let me know if you have any additional questions.

      Delete
    2. This comment has been removed by the author.

      Delete
  29. Thanks Kamilo! I was able to get data after running it as root on the mac mini. I thought I was the whole time, but I was mistaken. Thanks for the step by step. I still dont see any 514 or 1514 traffic in wireshark, but I guess thats another story.

    I am getting data from 'fios' and most of my dashboards are populating.
    Under Overview Dashboards >Network Event Overview is the only one that is NOT populating, in that category of dashboards.

    The others not populating are the Map of Connections , and the Bandwidth Overview is not populating (followed your instructions to enable the data input on this blog)

    I also wanted to note that i see nothing in Search>Data summary - i dont think thats correct.

    thanks in advance! love what i see so far!

    ReplyDelete