Adding DD-WRT Sourcetype

Sample Data

Here I'm going to show you how to add  a new source type into the Home Monitor App using dd-wrt as an example.  I'm planning on building a Splunk Technology Add-on (TA) for the Home Monitor app so that these extractions can be used by other apps or in a distributed environment.

First, let's take a look at the data.  We can see here that the data is already in Name = Value pairs with some extra fields that we can extract using the interactive Splunk Extraction tool.

2016-01-10 14:59:57 Kernel.Warning 192.168.28.1 Jan 10 06:59:57 kernel: ACCEPT IN=vlan2 OUT=br0 MAC=78:54:2e:4e:13:c9:00:17:10:85:ab:92:08:00:45:00:00:8f SRC=218.15.145.194 DST=192.168.28.57 LEN=143 TOS=0x00 PREC=0x00 TTL=43 ID=4934 PROTO=UDP SPT=14392 DPT=19598 LEN=123 MARK=0xa000
2016-01-10 14:59:57 Kernel.Warning 192.168.28.1 Jan 10 06:59:57 kernel: ACCEPT IN=br0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:1d:ba:67:d7:f2:08:00 SRC=192.168.28.11 DST=192.168.28.255 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=23255 PROTO=UDP SPT=137 DPT=137 LEN=58 MARK=0x35400
2016-01-10 14:59:57 Kernel.Warning 192.168.28.1 Jan 10 06:59:57 kernel: ACCEPT IN=vlan2 OUT=br0 MAC=78:54:2e:4e:13:c9:00:17:10:85:ab:92:08:00:45:00:00:84 SRC=123.26.105.194 DST=192.168.28.57 LEN=132 TOS=0x00 PREC=0x00 TTL=113 ID=15843 PROTO=UDP SPT=10538 DPT=19598 LEN=112 MARK=0xa000
2016-01-10 14:59:57 Kernel.Warning 192.168.28.1 Jan 10 06:59:58 kernel: ACCEPT IN=br0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:1d:ba:67:d7:f2:08:00 SRC=192.168.28.11 DST=192.168.28.255 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=23351 PROTO=UDP SPT=137 DPT=137 LEN=58 MARK=0x35400

I'm going to start by making this CIM compliant, so that means that this source will be able to populate the Home Monitor Dashboards without having to re-write the searches.

*** WARNING *** 

If you are going to build your own sourcetypes, please make all your changes in the $SPLUNK_HOME/etc/apps/homemonitor/local directory.  Making changes to the default directory will cause unnecessary grief since any updates I push will overwrite all your hard work.

Let's start with the Interactive Field Extractor.

First, let's add the sample data into Splunk.

Upload the sample file

Click on Save As 

See that the source type is now dd-wrt 

For testing purposes, create a test index and send this sample data into it

Let's Extract Fields

Notice that we are extracting fields for the source type dd-wrt.

Click on one of the events.

Select Regular Expression

Highlight 'ACCEPT' and name it action

Now you'll see the newly extracted field

Make sure to select permission for App

Now run a search and you'll see the newly created field.

Next step, let's add some logic to the props.conf : Go ahead and open your $SPLUNK_HOME/etc/apps/homemonitor/local/props.conf

You should see this entry [dd-wrt] :

[dd-wrt]
pulldown_type = 1
EXTRACT-action = (?i) .*?: (?P<action>\w+)(?= )

Now, let's 'normalize' the fields for Source IP, Source Port, Destination IP and Destination Port.

FIELDALIAS-dst = DST as dest_ip
FIELDALIAS-dpt = DPT as dest_port
FIELDALIAS-proto = PROTO as protocol
FIELDALIAS-SPT = SPT as src_port
FIELDALIAS-SRC = SRC as src_ip
EVAL-direction = if(match(OUT,"eth*"), "out", "in")

This stanza will create a new field called direction and base the direction on what interface is being used by the field 'OUT.' If the OUT field has eth0, then it will determine the direction as 'out,' otherwise it will set the direction to 'in.' 

Finally, we can add some lookups to enrich the data. The first lookup normalizes the action to either ACCEPT or BLOCK. The second lookup actually does reverse lookup for IP addresses. 

LOOKUP-action_lookup = action_lookup action OUTPUT action2

LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host

The final props.conf entry should look like this :

[dd-wrt]
pulldown_type = 1
EXTRACT-action = (?i) .*?: (?P<action>\w+)(?= )
FIELDALIAS-dst = DST as dest_ip
FIELDALIAS-dpt = DPT as dest_port
FIELDALIAS-proto = PROTO as protocol
FIELDALIAS-SPT = SPT as src_port
FIELDALIAS-SRC = SRC as src_ip
EVAL-direction = if(match(OUT,"eth*"), "out", "in")
LOOKUP-action_lookup = action_lookup action OUTPUT action2
LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host

I will be adding this to the default props.conf for the next release of Home Monitor 4.4.3.