Adding Quantam Sourcetype
props.conf
transforms.conf
For whatever reason, when I paste this into the comments bar it does not properly show the '<action>' extraction.
[syslog]
TRANSFORMS-changesourcetype = asus, fios, link sys, mikro, netgear, openwrt, pfsense, quantum, sophos, skyhub, tomato
[quantum]
FIELDALIAS-dst = DST as dest_ip
FIELDALIAS-dpt = DPT as dest_port
FIELDALIAS-proto = PROTO as protocol
FIELDALIAS-SPT = SPT as src_port
FIELDALIAS-SRC = SRC as src_ip
EXTRACT-action = ^[^\]\n]*\]:\s+(?P<action>\w+)
EVAL-direction = if(match(OUT,"eth*"), "out", "in")
pulldown_type = 1
LOOKUP-action_lookup = action_lookup action OUTPUT action2
LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host
transforms.conf
[quantum]
# Make sure that this matches the hostname of your router, quantum is just an example.
REGEX = quantum
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::quantum
DEST_KEY = MetaData:Sourcetype
For whatever reason, when I paste this into the comments bar it does not properly show the '<action>' extraction.
Hey,
ReplyDeleteSo the errors stopped! That definitely did the trick for splunk complaining on startup. However, all graphs, panels, etc. are all still "No Results Found", "0", or "Waiting for Input" except for the Total Number of Events from the Home Network Overview (which does change based on the time query). I have validated that the router is sending the data to syslog, and that data is getting into splunk (index=homemonitor). Below are props.conf and transforms.conf:
/opt/splunk/etc/apps/homemonitor/local:cat props.conf
[syslog]
TRANSFORMS-changesourcetype = quantum
[quantum]
FIELDALIAS-dst = DST as dest_ip
FIELDALIAS-dpt = DPT as dest_port
FIELDALIAS-proto = PROTO as protocol
FIELDALIAS-SPT = SPT as src_port
FIELDALIAS-SRC = SRC as src_ip
EXTRACT-action = ^[^\]\n]*\]:\s+(?P\w+)
EVAL-direction = if(match(OUT,"eth*"), "out", "in")
pulldown_type = 1
LOOKUP-action_lookup = action_lookup action OUTPUT action2
LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host
/opt/splunk/etc/apps/homemonitor/local:cat transforms.conf
[portlookuptable]
filename = application_protocals.csv
[quantum]
# Make sure that this matches the hostname of your router, fios is just an example.
REGEX = 192\.168\.136\.1
#For the hostname I am just using the IP as that is what shows up in Splunk - there is no DNS entry for this IP. Since it is a Regex field, I am using the "\" in front of "." although I have also tried it without the "\" with the same results.
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::quantum
DEST_KEY = MetaData:Sourcetype
FORMAT = homemonitor
DEST_KEY = _MetaData:Index
#Lookup normalizing Action
[action_lookup]
filename = action_lookup.csv
[dnsLookup]
external_cmd = external_lookup.py host ip
fields_list = host, ip
And, since this file was also changed, here is the action_lookup.csv:
cat ../lookups/action_lookup.csv
action,action2
BLOCK,BLOCK
ACCEPT,ACCEPT
pass,ACCEPT
block,BLOCK
DROP,BLOCK
Accepted,ACCEPT
Blocked,BLOCK
I am probably missing something so stupid ... but for the life of me I cannot see what it is. Any ideas? Thanks again!
Let's do some simple searches and make sure the data is properly coming into the app. First, run this search :
Deleteindex=homemonitor sourcetype=quantum | stats count by action2
(This will give you the total number of accepts and blocks that your router has logged.)
If this search does not populate, then we have to make sure that the REGEX is working properly to extract the action field.
Next, let's run this search :
index=homemonitor sourcetype=quantum | stats count by direction
If that comes up with no results, then run this search instead
index=homemonitor sourcetype=quantum |eval direction=if(match(match(OUT,"eth*"), "out", "in") | stats count by direction
This should give you a breakdown of in vs. out bound traffic.
The only issue you're probably running into now is the field extractions, these searches will just help narrow down where the problem might be and which fields to extract.
Thanks,
Kam
Kam, Strange -- even though we are telling it to use a sourcetype of quantum, they are showing up with a sourcetype of syslog. Whatever, just odd.
DeleteNarrowed all queries to a 30 minute window.
index=homemonitor sourcetype=syslog | stats count by action2
440 of 440 events matched, but no results in current time range.
index=homemonitor sourcetype=syslog | stats count by direction
440 of 440 events matched, but no results in current time range.
index=homemonitor sourcetype=syslog |eval direction=if(match(OUT,"eth*"), "out", "in") | stats count by direction
in: 120
out: 300
Thanks!
The reason it's not picking up the source type is that it's trying to match the hostname and you might have set it to DNS instead of IP. Regardless, you can either manually override the input or just change the REGEX stanza to be the hostname of your router in the transforms.conf.
DeleteOnce you get this to work, then the extractions will happen and the dashboards should populate. Check out the wiki (https://github.com/amiracle/homemonitor/wiki) and see the section called Configuring the App.