Setting up Verizon FiOS Router for Home Monitor **UDPATED**

Log into your router and select Advanced, click Yes to proceed.

Select "System Settings"
Scroll down and enable System Logging and Security Logging.  Next, Enter the IP Address of your Splunk Server.

Now select the Firewall Settings
Click on the Security Log
Click on the Settings button.

Put the check box on all the items you want logged in Splunk, then click Apply.


Now log into your Splunk instance and go to the Manager.


In Splunk, select Add Data and then select Data Inputs.

Click on UDP

Follow each step and MAKE SURE to click on the check box for More settings. When complete, click Save.

When done, your Data inputs page should look like this.

Comments

  1. The homemonitor index was not automatically created when installing the app. I had to manually create it.

    ReplyDelete
    Replies
    1. Just fixed it in the latest version 1.2. Thanks and good catch!

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. On your Splunk server, can you do a tcpdump and look for traffic on udp port 514?

    ReplyDelete
  4. No I'm on Windows 8. I went in and allowed port 514 UDP through the firewall and still no dice.

    ReplyDelete
  5. I would run wireshark on your Splunk instance and validate that you are seeing traffic from your FiOS router. You can also validate if Splunk is receiving data by taking a look at the homemonitor index. Search index=homemonitor * All Time.

    ReplyDelete
  6. I can see traffic... homemonitor index event count shows only 15 from yesterday all with the following:

    Jan 15 22:58:21 192.168.1.1 Jan 15 23:58:20 2013 Wireless_Broadband_Router Unknown PTR name format
    host=192.168.1.1 Options| sourcetype=syslog Options| source=syslog Options

    ReplyDelete
  7. This could mean a couple of things. First, what version router do you have? (Model number and firmware version). Next, what options did you select on your routers configuration page, Information, Warn, or Error?

    ReplyDelete
  8. Router Info:

    Model Name: MI424WR-GEN3I
    Firmware Version: 40.19.36
    Hardware Version: I

    Both System and Security logging are set to information

    ReplyDelete
  9. In your Splunk instance, go to Manager->Data Inputs->UDP Click on 514 and when it opens, make sure that "Set Sourcetype" is either set to "From a List" and syslog is selected, or you can use "Manual" and type in syslog. Let me know if that works.

    ReplyDelete
  10. The first thing I would do is just do a search in Splunk like this: index=homemonitor and go ahead and do it for All Time. If you see results, then the data is properly flowing into the index. If not, we need to revisit your setup. Are you running your Splunk server on Windows or Linux (Mac OS X)?

    ReplyDelete
  11. Did you try running tcpdump on your Splunk box to validate that you're receiving traffic from the FiOS router? (tcpdump -i eth0 UDP) Also, is there a reason you're not running Splunk as root on your Ubuntu box?

    ReplyDelete
  12. What does the output look like when you run index=homemonitor in Splunk?

    ReplyDelete
  13. I figured out what the problem was with the app. It's a simple fix, you just have to go into your Data Inputs (Settings -> Data Inputs -> UDP -> click on the 514). Change the sourcetype from syslog to fios. That will make all your dashboards populate. I've been working on something that will fix that in a later release. Good catch!

    ReplyDelete
  14. I just updated the app to version 3.0.4 that has all the fixes listed above. Please feel free to download it and test it out with your system. Thanks for pointing out the issues that were in the app.

    ReplyDelete
  15. Let's try and edit the panels and see if we can get them to populate with your data. First, let's make sure you're getting the data with the correct sourcetype. Go into search and type: index=homemonitor | stats count by sourcetype

    You can do it for the last 15 minutes.

    You should see the sourcetype fios and no syslog.

    Send me an email, kamilo@gmail.com and we can do a quick webex or Google Hangout.

    ReplyDelete
  16. Home monitor and pfsense 2.2.1. The search arguments in home monitor are not compatible with the input from pfsense. I have lots of input from pfsense but the surch arguments in the panels. Give zero hits.

    ReplyDelete
  17. This comment has been removed by the author.

    ReplyDelete
  18. I have a vanilla install of Splunk on a Macbook Pro. I have only installed Stream and Home Monitor 4.0.1. I have reviewed the installation video on youtube for version 2.x and read the blogs. I followed the installation instructions provided. The router was configured to forward syslog to my laptop and I can verified 514 traffic is being received with tcpdump. I created the UDP Data Input, but didn't see FIOS as a source type from the pick list. It's set for syslog. I'm not sure what I'm missing

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. This comment has been removed by the author.

      Delete
  19. I have a vanilla install of Splunk inside a CentOS7-vm on my Macbook Pro, I have fusion8pro installed there where the VM runs. I have followed the directions above to the T and I am not seeing anything in the dash board. I have configured the inputs to account for up\514 and I did select the index. One nuance I did notice was that a UDP connection for syslog was in data inputs. So I had to modify that one for the advanced settings. no clue how to fix, would love to see the data......

    ReplyDelete
  20. I'm working on a video that will walk you through the install of Splunk, the home | monitor > app and get the data into Splunk from your router / modem.

    The easiest way to fix the input is to go into the $SPLUNK_HOME/etc/apps/homemonitor/default and copy the inputs.conf file to the $SPLUNK_HOME/etc/apps/homemonitor/local directory and make the modification there. Once the changes have been made (change the port or whatever change you want to make) then it should start receiving the data.

    Make sure that you've either disabled iptables (sudo service iptables stop) on your Linux server OR just add an entry for UDP 514 in-bound (modify /etc/sysconfig/iptables/). Let me know if that works and if you don't see any data.

    ReplyDelete
  21. Can you add support for D-Link router, mine is DIR-655. thanks

    ReplyDelete
  22. I am familier with Splunk but new to Home Monitor App. First of all your app is amazing, thank you.

    So I got my dd-wrt log to populate and its working great. Then I moved to fios router. I noticed there is no sourcetype where it was mentioned in earlier comments. Do I need to create that sourcetype and if so is the parsing parameters documented somewhere, I couldn't find it?

    Currently using Splunk 6.3.3 and Home Monitor 4.4.2

    Thank you in advance!

    ReplyDelete
    Replies
    1. Yebro,

      Thanks for downloading and using the app. If you want to change back your sourcetype, you can either do this via the Web GUI or modifying a configuration file using the CLI. The WebUI option is good if you're going to stick with your FIOS sourcetype; however if you’re going to be going back and forth between different routers, it might make more sense to make the change to the configuration files.

      The WebUI option is simple, just go into Settings -> Data Inputs -> UDP 514 and then over ride the source type as fios.

      The CLI option is a more scalable solution if you’re going to be jumping around from different sources. First, get the hostname for your router. For this example, I’m going to use “home.fios” as the hostname. To make the change, log into your Splunk server and go to $SPLUNK_HOME/etc/apps/homemonitor/local and edit the transforms.conf file. The transforms.conf file typically does not exist in this directory, just simply create it and add the following entry. Under the REGEX = field, just make sure that you set the entry to your hostname.

      [fios]
      # Make sure that this matches the hostname of your router, fios is just an example.
      REGEX = home.fios
      SOURCE_KEY = MetaData:Host
      FORMAT = sourcetype::fios
      DEST_KEY = MetaData:Sourcetype

      Once you’re done making the changes, go ahead and restart your Splunk instance. Now, let’s say you want to add a dd-wrt router (hostname = ddwrt.home), you can add this line to the same transforms.conf file :

      [dd-wrt]
      # Make sure that this matches the hostname of your router, fios is just an example.
      REGEX = ddwrt.home
      SOURCE_KEY = MetaData:Host
      FORMAT = sourcetype::dd-wrt
      DEST_KEY = MetaData:Sourcetype

      Now, anytime a machine sends data in as syslog then Splunk will look at the hostname. If it matches ddwrt.home, it will set the source type to dd-wrt; or if it’s home.fios then it will set the source type to fios all automatically.

      I know it’s a long winded answer, but that should cover you. Let me know if you have any additional questions.

      Thanks,
      Kam

      Delete
    2. Thank you Kam,

      As for props.conf file, how would the fios parsing parameters look like. I wasn't able to see "fios" as a sourcetype however when I tried creating it, it said I was already created.
      Thank you again

      Delete
    3. If you want to see how the fields are created for the fios sourcetype, go to $SPLUNK_HOME/etc/apps/homemonitor/default/props.conf . Don't modify the file there, any changes will be overwritten when I make any updates.

      Delete
    4. Thank you Kam,

      So I tried using those fields for my fios box but did not match. Here are few examples from my fios box:

      Feb 17 00:22:29 192.168.1.1 Feb 17 00:22:28 2016 fios IN: ACCEPT [57] Connection closed ( : UDP 8.8.8.8:53 <-->8.8.8.8:53 [192.168.1.2:59059] br0 Route Incoming UNSECURED )

      Feb 17 00:22:12 192.168.1.1 Feb 17 00:22:11 2016 fios OUT: ACCEPT LAN-OUTBOUND [38] Wireless Broadband Router initiated traffic (UDP 100.29.130.190:1024->68.254.0.13:53 on eth1)

      Feb 17 00:22:12 192.168.1.1 Feb 17 00:22:11 2016 fios Empty name

      Delete
    5. Got it to work. I know not to tweak this file /opt/splunk/etc/apps/homemonitor/default/props.conf but I noticed no way to get it working with current config. so here are my changes/correction.

      [fios]
      DATETIME_CONFIG =
      NO_BINARY_CHECK = true
      category = Custom
      disabled = false
      pulldown_type = true
      EXTRACT-direction = ^(?:[^ \n]* ){9}(?P[^:]+)
      EXTRACT-protocol = ^[^\(\n]*\(\s+:\s+(?P\w+)
      EXTRACT-action = ^(?:[^ \n]* ){10}(?P\w+)
      EXTRACT-state = ^[^\]\n]*\]\s+(?P\w+\s+\w+)
      EXTRACT-src_ip = ^[^\(\n]*\(\s+:\s+\w+\s+(?P\d+\.\d+\.\d+\.\d+)
      EXTRACT-src_port = ^(?:[^:\n]*:){7}(?P\d+)
      EXTRACT-nat_ip = ^[^>\n]*>(?P\d+\.\d+\.\d+\.\d+)
      EXTRACT-dest_ip = ^(?:[^\[\n]*\[){2}(?P\d+\.\d+\.\d+\.\d+)
      EXTRACT-dest_port = ^(?:[^:\n]*:){9}(?P\d+)
      EXTRACT-interface = ^(?:[^\]\n]*\]){2}\s+(?P[^ ]+)
      EXTRACT-reason = ^[^\[\n]*\[(?P\d+)
      EXTRACT-config_change_user = ^[^\(\n]*\(\w+\s+\w+\s+(?P\w+)
      LOOKUP-fios = action_lookup action OUTPUTNEW action2
      LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host

      Delete
  23. This comment has been removed by the author.

    ReplyDelete
  24. That's great, if you want just copy that stanza to your $SPLUNK_HOME/etc/apps/homemonitor/local/props.conf file. This way when I make an update it will not overwrite your changes.

    ReplyDelete
  25. Hello Kamilo,
    Is google maps no longer available via Splunk Apps or add ons? I'm not longer able to find it through splunk enterprise, the latest version 6.3.3. I was trying to install it as instructed by the video. Any insight?

    thanks in advance!

    ReplyDelete
    Replies
    1. I think I just answered my own question, It's not available in Splunk Enterprise 6.x - so that makes me wonder about home monitor and the latest 6.x enterprise version and mapping....just wondering if you're now using built in splunk maps or some other method? keep up the good work!

      Delete
    2. Yes I started using the defaul maps that come with Splunk enterprise.

      Delete
    3. So when you say default maps that come with Splunk Ent, that means that i do NOT have to download other apps to install correct? I cannot get any of the maps to work unfortunately. My input is fios, when i click on Map of Connections.....just sitting stale on Waiting for input.... is there any trick to getting these maps to populate on that dashboard? any files to manipulate? thanks in advance.

      Delete
  26. Kamilo,
    appologies, reposting as its own post now.
    just wondering if there is any update to Craigs issue, I didn't see a remedy/response in this thread. I've been poking around many related threads and I'm still having issues.

    I have a similar problem, I'm using the latest version of Splunk Enterprise (6.3.3) on a mac mini, El Capitan, with the latest version of Home Monitor (4.4.2).

    I've been reading several blogs and searched this one and the videos. That said, I think I have it correctly configured, please let me know if i'm way off; In Data Input - I've configured the UDP 514 as instructed, but with the update of using 'fios' in the place of 'Source name override' , set source type manual, picked from the list 'syslog', then I checked the more settings, and for host, selected the button for custom. I've tried whatever popped up as default here, left it blank, and since one of your posts says use the router hostname (i take it that that means the fios host name, your router hostname, found in Fios system settings as 'Wireless System Hostname' in which i put as 'fios')
    note from https://4dd0p3r470r.wordpress.com/2015/01/31/how-to-splunk-home-monitor-for-fios-routers/
    I followed this as well as this was a little vague in repeated postings: IMPORTANT UPDATE(!): in the field “Source name override” put “fios” instead of the “syslog” - i did this - let me know if thats wrong too.

    which leaves the final thing in the Data set , i chose homemonitor as the index, and hit save....

    unfortunately, nothing is populating in my homemonitor. Not sure what I'm doing wrong here.

    I also saw this thread; https://answers.splunk.com/answers/310324/home-monitor-how-to-configure-a-netcomm-modem-rout.html#answer-310832

    in which you answer about Mac osx , 'If you're running Splunk on Linux (or Mac OS X), then you'll either have to run Spunk as root or change the port that your modem is sending syslog to 1514 UDP.'
    -I assume I am running as root, im the Admin on the machine, i even enabled the root user, but not sure if thats an issue with simply executing the Splunk icon when i launch it. so I think it would run as me being the admin which should be root - let me know if i'm mistaken.
    'change the port that your modem is sending syslog to 1514 UDP' - I assume that you are saying change your Fios router to send the syslog via port 1514? I cannot figure out how to change my Fios router to change that......(i also tried changing the Data Set back and forth to 514 and 1514 via the web UI and tried hard coding into the .conf files) niether of which made a difference.....still no traffic coming to the splunk server from Fios router.

    I also used wireshark to check for anything UDP coming from my Fios router, and i havent seen a single packet for hours using 514 and 1514, i even cleared my logs and started them over to generate current traffic. My NTP is synched too on fios and splunk.

    I guess that might explain why i cant get home monitor to show anything as well on Data Summary and trying in the search index=homemonitor sourcetype=fios, and index=homemonitor | stats count by sourcetype

    I've burned 8 hours on this today - really not sure what else to try....any ideas? by the way, great work! I really love what it can do and cant wait to get it working! thanks in advance.

    ReplyDelete
    Replies
    1. First, when you run Splunk you are probably not running it as root. In order to run splunk as root you will have to go into your terminal and type sudo "$SPLUNK_HOME/bin/splunk start" . That will start splunk as root on your machine. If you want to have it start automatically as root, you will need to run "sudo $SPLUNK_HOME/bin/splunk enable boot-start -user root"

      I've used the environment variable $SPLUNK_HOME, if it's in your profile, then you should be fine to run the commands as listed above, otherwise go to your splunk home directory (/Applications/Splunk I assume) and run sudo bin/splunk start . It will prompt you for your admin password and then run the command.

      Once you've finished that, then you should start seeing data come into your Mac on port UDP 514.
      (http://docs.splunk.com/Documentation/Splunk/latest/Installation/InstallonMacOS)

      Let me know if you have any additional questions.

      Delete
    2. This comment has been removed by the author.

      Delete
  27. Thanks Kamilo! I was able to get data after running it as root on the mac mini. I thought I was the whole time, but I was mistaken. Thanks for the step by step. I still dont see any 514 or 1514 traffic in wireshark, but I guess thats another story.

    I am getting data from 'fios' and most of my dashboards are populating.
    Under Overview Dashboards >Network Event Overview is the only one that is NOT populating, in that category of dashboards.

    The others not populating are the Map of Connections , and the Bandwidth Overview is not populating (followed your instructions to enable the data input on this blog)

    I also wanted to note that i see nothing in Search>Data summary - i dont think thats correct.

    thanks in advance! love what i see so far!

    ReplyDelete
  28. This comment has been removed by the author.

    ReplyDelete
  29. Hey Kam, I really love your app and after reading some posts I have got Most of it working except for the fios router config dashboard, not sure what I am doing wrong, pretty much everything else is working

    ReplyDelete
  30. Depending on what Verizon did they might have made some changes to the way the app can extract the fields necessary for the dashboards.

    I'd recommend looking at the extractions in the props.conf and seeing which ones work and fix the ones that don't. If you look on the wiki for my github, it shows how to troubleshoot adding extractions.

    Let me know if that helps.

    Kam

    ReplyDelete
  31. Well, I have done alot, ill keep looking for something that relates to the state. Another issue I am having is on the home network overview and bandwidth overview I cant get the bandwidth panels with the avg upload or download or ping to work. I changed the python script to what you have on github but still nadda.

    ReplyDelete
  32. urls = [
    'https://www.speedtest.net/speedtest-servers.php',
    '://www.speedtest.net/speedtest-servers-static.php',
    '://www.speedtest.net/speedtest-servers.php',
    '://c.speedtest.net/speedtest-servers-static.php',
    '://c.speedtest.net/speedtest-servers.php',
    ]

    ReplyDelete
  33. Any ideas on the issue with the bandwidth overview yet?

    ReplyDelete
  34. Kam, I'm trying to get the app to work for the first time. I believe I have the router and application configured correctly but I do not see any data. I'm using Windows10 and have configured my router to log to the local machine running Splunk. I've set UDP source name to fios and alternatively to syslog. I've reviewed the blog and youtube video but still can't get it to work. Could it be an issue with a windows10 setting?
    Thanks, Gilbert

    ReplyDelete
  35. Hello Kamilo,

    Looks like I have a different Fios log format, as such the dashboards are not populating. This seems to be caused by the "Destination" field is not being properly extracted. My 2 log formats ("Accepted" and "Blocked"), captured in SPLUNK are as follows:

    Accepted:
    12/29/16
    10:11:40.000 AM
    Dec 29 10:11:40 192.168.0.1 ulogd[582]: Accepted IN=br-lan OUT=eth0 MAC=11:1d:11:1f:11:11:11:f1:1f:ce:11:a1:11:11 SRC=192.168.0.152 DST=172.111.1.11 LEN=40 TOS=00 PREC=0x00 TTL=127 ID=21567 DF PROTO=TCP SPT=43573 DPT=443 SEQ=3940399687 ACK=2770573242 WINDOW=255 ACK URGP=0 MARK=2

    Blocked:
    12/29/16
    10:10:39.000 AM
    Dec 29 10:10:39 192.168.0.1 ulogd[582]: Blocked IN=eth0 OUT= MAC=11:1d:11:1f:11:11:11:f1:1f:ce:11:a1:11:11 SRC=175.117.3.42 DST=192.168.0.51 LEN=40 TOS=00 PREC=0x00 TTL=58 ID=354 PROTO=TCP SPT=443 DPT=45784 SEQ=2546652287 ACK=0 WINDOW=0 RST URGP=0 MARK=0

    From what I see is that "Accepted IN" is outbound traffic on br-lan. And "Blocked IN" is inbound traffic on eth0. Any suggestions on updating the Destination REGEX?

    Thanks

    ReplyDelete
  36. I'm getting bandwidth populated but that's it; running on mac. one thing to note is when running the root command in terminal i'm getting, "sudo: /bin/splunk: command not found" could that be the problem?

    ReplyDelete
    Replies
    1. To validate that the script works, go to the splunk home directory (where you installed Splunk) and then navigate to the etc/apps/homemonitor/bin directory. Once there, you should be able to run the speedtest.sh command and see some results:

      mac-mbp:bin$ sh speedtest.sh
      2017-01-19 17:11:51
      Ping=27.083 ms
      Download=100.19 Mbit/s
      Upload=47.13 Mbit/s

      This also assumes that the SPLUNK_HOME environment variable has been defined, which is the case if you installed splunk. Just run 'env' command to see the variables.

      Delete
  37. Great post! I got it up and running and with Splunk not as root (which is recommended by Splunk). One just needs to set up a rule to forward 514 to say 5514. So, for Centos7x64 this would be:

    firewall-cmd --zone="public" --add-forward-port=port=514:proto=udp:toport=5514

    Now going through all the other Help stuff to get the other widgets working.

    ReplyDelete
  38. Just installed splunk 6.6.2 on my Mac and installed home monitor to look at FiOS router. I tried to follow your setup above, but I'm not getting any data into splunk. Note with 6.6.2 when I configure the UDP setting and select more settings the input to set address of the router is not available unless unless I select custom. Tried that, but I'm still no seeing data.
    Reading about I saw you suggested executing the speedtest as a test. The SPLUNK_HOME ENV wasn't set although I did install splunk. I set SPLUNK_HOME and ran the speedtest. Received below. Suggestions?
    2017-07-19 09:12:23
    Ping=9.973 ms
    Download=57.12 Mbit/s
    Upload=37.00 Mbit/s

    ReplyDelete
  39. I'm new to Splunk and figured Home Monitor would be a good starting point... I have a HomeLab running on Windows2016 Standard for the VMworkstation, Splunk and a Bastion Host. A pfSense v2.3.4 as a VM, I have a Splunk Ent v7.1.1 running in a VM, I installed Home Monitor 4.5.1 and configured it per the video and Blog... but no Data in the Index="homemonitor". I set up Kiwi syslog server on a third system and had pfSense configured to send syslogs to both the Splunk Ent and to the Kiwi system. So I'm getting logs to the Kiwi server and to confirmed that syslogs are getting to the Splunk system via Wireshark. Is it phSense vs FIOS or is there something else you could suggest?

    ReplyDelete
    Replies
    1. So after going back to Ent v6.2.3 I was able to get data into the index=homemonitor. :-) But all the fields in "Home Network Overview" show "Waiting For input or data", Any thought on how I should proceed?

      Delete
  40. First question is did you setup Splunk to listen on UDP 514? By default Splunk is listening to that port once you enable it via the UI (Settings -> Data Inputs ->UDP->514 enabled). If this is a Linux based machine then you’ll need to be root to enable that port. Another option is to setup a Splunk forwarder to send data from your syslog server into Splunk.

    I actually have both pfsense and FiOS quantum data flowing into Splunk via a syslog (rsyslog) server and a Splink forwarder.

    Let me know if that helps, and we can work on it from there.

    ReplyDelete
    Replies
    1. So after going back to Ent v6.2.3 I was able to get data into the index=homemonitor. :-) But all the fields in "Home Network Overview" show "Waiting For input or data", Any thought on how I should proceed?

      Delete
    2. So the pfSense FW has "Remote Syslog Contents" has Everything checked,

      Delete
  41. All you need to do now is pick a sourcetype (pfsense, quantum etc.) and the dashboards should populate assuming there is data there.

    ReplyDelete

Post a Comment

Please do not post ads or spam to this comment section.

Popular Posts