tag:blogger.com,1999:blog-49350856601302831802024-02-19T09:01:16.253-08:00Splunk Apps by KamThis is my simple blog that follows the posts for the app I developed for the Splunk platform. amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.comBlogger20125tag:blogger.com,1999:blog-4935085660130283180.post-21903015614332786882016-02-11T09:24:00.001-08:002016-02-11T09:24:12.022-08:00Adding DD-WRT Sourcetype<div class="tr_bq">
Sample Data</div>
<br />
Here I'm going to show you how to add a new source type into the Home Monitor App using dd-wrt as an example. I'm planning on building a Splunk Technology Add-on (TA) for the Home Monitor app so that these extractions can be used by other apps or in a distributed environment.<br />
<br />
First, let's take a look at the data. We can see here that the data is already in Name = Value pairs with some extra fields that we can extract using the interactive Splunk Extraction tool.<br />
<blockquote>
2016-01-10 14:59:57 Kernel.Warning 192.168.28.1 Jan 10 06:59:57 kernel: ACCEPT IN=vlan2 OUT=br0 MAC=78:54:2e:4e:13:c9:00:17:10:85:ab:92:08:00:45:00:00:8f SRC=218.15.145.194 DST=192.168.28.57 LEN=143 TOS=0x00 PREC=0x00 TTL=43 ID=4934 PROTO=UDP SPT=14392 DPT=19598 LEN=123 MARK=0xa000<br />
2016-01-10 14:59:57 Kernel.Warning 192.168.28.1 Jan 10 06:59:57 kernel: ACCEPT IN=br0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:1d:ba:67:d7:f2:08:00 SRC=192.168.28.11 DST=192.168.28.255 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=23255 PROTO=UDP SPT=137 DPT=137 LEN=58 MARK=0x35400<br />
2016-01-10 14:59:57 Kernel.Warning 192.168.28.1 Jan 10 06:59:57 kernel: ACCEPT IN=vlan2 OUT=br0 MAC=78:54:2e:4e:13:c9:00:17:10:85:ab:92:08:00:45:00:00:84 SRC=123.26.105.194 DST=192.168.28.57 LEN=132 TOS=0x00 PREC=0x00 TTL=113 ID=15843 PROTO=UDP SPT=10538 DPT=19598 LEN=112 MARK=0xa000<br />
2016-01-10 14:59:57 Kernel.Warning 192.168.28.1 Jan 10 06:59:58 kernel: ACCEPT IN=br0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:1d:ba:67:d7:f2:08:00 SRC=192.168.28.11 DST=192.168.28.255 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=23351 PROTO=UDP SPT=137 DPT=137 LEN=58 MARK=0x35400 </blockquote>
<div>
I'm going to start by making this CIM compliant, so that means that this source will be able to populate the Home Monitor Dashboards without having to re-write the searches.</div>
<div>
<br /></div>
<div style="text-align: center;">
<b>*** WARNING *** </b></div>
<div>
If you are going to build your own sourcetypes, please make all your changes in the $SPLUNK_HOME/etc/apps/homemonitor/local directory. Making changes to the default directory will cause unnecessary grief since any updates I push will overwrite all your hard work.</div>
<div>
<br /></div>
<div>
Let's start with the Interactive Field Extractor.</div>
<div>
<br /></div>
<div>
First, let's add the sample data into Splunk.</div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiygyQFhaWNtGI_BTz4fHAYL4HCjnOpYGCsVSfYTwnQ95VPqRl4SDQI-i-Gx7j46zS6a3Ya2GsGPJ1eEj4iDeF6mlMTR3xdwBEjVvSmf2xZYw7wckoa0YcVS5tK_5iVSu0GyTAlZO1H1X7_/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiygyQFhaWNtGI_BTz4fHAYL4HCjnOpYGCsVSfYTwnQ95VPqRl4SDQI-i-Gx7j46zS6a3Ya2GsGPJ1eEj4iDeF6mlMTR3xdwBEjVvSmf2xZYw7wckoa0YcVS5tK_5iVSu0GyTAlZO1H1X7_/s640/1.png" width="640" /></a></div>
<div>
Upload the sample file</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLxrplkDK-O6BVZQ2iC2xnzJeMwLFFPUOcs-hAoJzxP5z7ZqIXX2EqTIA-7fKKNXVPndOyad3ohFYCp6Ecr86MSb3cUDstGwvr1X2g7K9VSmYmlATQ2zNPdjoV7NKP4w240hxVzAh5QmBu/s1600/2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="162" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLxrplkDK-O6BVZQ2iC2xnzJeMwLFFPUOcs-hAoJzxP5z7ZqIXX2EqTIA-7fKKNXVPndOyad3ohFYCp6Ecr86MSb3cUDstGwvr1X2g7K9VSmYmlATQ2zNPdjoV7NKP4w240hxVzAh5QmBu/s640/2.png" width="640" /></a></div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-XWXTrq000tjAXVaSEEpQgOtsjZ2LNEC2YbeR0Lo0YLuvxf-xK_1Tre1W8KPNvO1ZFCpyN7GX9_x4A3IdvhtA8muY0SwWmZ3aOEBgq_WNgX4AzP5Gu97hnbcdEFwoVOXJqF3uhacrAWT7/s1600/3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="390" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-XWXTrq000tjAXVaSEEpQgOtsjZ2LNEC2YbeR0Lo0YLuvxf-xK_1Tre1W8KPNvO1ZFCpyN7GX9_x4A3IdvhtA8muY0SwWmZ3aOEBgq_WNgX4AzP5Gu97hnbcdEFwoVOXJqF3uhacrAWT7/s640/3.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiE0MqK7Sx2NryNYjQhfTnSkrSNULEJkv57QqrrDc6blXpgw7rBZ7gHZX7LAQk5SC6v_LQBDdI9ncMhEd5Nwqx5dIcJDKMvhWfWNFceAcPzugJsaMMcVqxt4VwaCm5yCQVPvZhm3eQ7CHVM/s1600/4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="384" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiE0MqK7Sx2NryNYjQhfTnSkrSNULEJkv57QqrrDc6blXpgw7rBZ7gHZX7LAQk5SC6v_LQBDdI9ncMhEd5Nwqx5dIcJDKMvhWfWNFceAcPzugJsaMMcVqxt4VwaCm5yCQVPvZhm3eQ7CHVM/s640/4.png" width="640" /></a></div>
Click on <b>Save As </b><br />
<b><br /></b></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL-R4_vksY8ATq3Vj1QgyPmJhoLLalqEu6ePSdgsHZiC5X6NQgLt7lWqox7Lfbz7FY41jFETPogdTN4N_aHRuS2Pz-dkTsj9TxUbagRcEtYJaA14CBNAoUu3lM7h5o38ZjfYOkgz9q8nge/s1600/5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL-R4_vksY8ATq3Vj1QgyPmJhoLLalqEu6ePSdgsHZiC5X6NQgLt7lWqox7Lfbz7FY41jFETPogdTN4N_aHRuS2Pz-dkTsj9TxUbagRcEtYJaA14CBNAoUu3lM7h5o38ZjfYOkgz9q8nge/s640/5.png" width="640" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEfaid5W5xME7sAVFcKHPQuMj5rSdVPOGa9rsxGf5wQ_NvZ9Joo6GQiyR7tEc_5yE85XHMtDfY1fi3HKwzVAFvQlg5TiG-Qe4doUkQwjpgUTkaQw-SVMr5HF3i7ER0qOf0bVtWjXPpJJbF/s1600/6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="206" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEfaid5W5xME7sAVFcKHPQuMj5rSdVPOGa9rsxGf5wQ_NvZ9Joo6GQiyR7tEc_5yE85XHMtDfY1fi3HKwzVAFvQlg5TiG-Qe4doUkQwjpgUTkaQw-SVMr5HF3i7ER0qOf0bVtWjXPpJJbF/s320/6.png" width="320" /></a></div>
<b><br /></b></div>
<div>
See that the source type is now <b>dd-wrt </b></div>
<div>
<b><br /></b></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLGADnsAJkWGPwueUnRBJ-xJULhAFxIrznJ95-ISg2phJXhTV7jsHYT7kuF1wju3NFSbxLOcYCaC1mSgN_Uv-7lC4LUXntMMEC8jw4SiwWT0BXPHtC2DGzyNanMhgLfqP3bvaGiAjy3VGN/s1600/7.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="366" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLGADnsAJkWGPwueUnRBJ-xJULhAFxIrznJ95-ISg2phJXhTV7jsHYT7kuF1wju3NFSbxLOcYCaC1mSgN_Uv-7lC4LUXntMMEC8jw4SiwWT0BXPHtC2DGzyNanMhgLfqP3bvaGiAjy3VGN/s640/7.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
For testing purposes, create a <b>test index</b> and send this sample data into it<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJxtQ5EC1wAO9cXT0bw9KeSi4lhtiSe0i-OIceE0Oo51aGcysJPAsIa6dor5P-q77LjeZdw37P98EldwhnprvIj1hEqgqysbJoiJ6x4OzRz74nfJreKkl92TJfcieuty_4M94MJWUkRll-/s1600/8.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="316" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJxtQ5EC1wAO9cXT0bw9KeSi4lhtiSe0i-OIceE0Oo51aGcysJPAsIa6dor5P-q77LjeZdw37P98EldwhnprvIj1hEqgqysbJoiJ6x4OzRz74nfJreKkl92TJfcieuty_4M94MJWUkRll-/s640/8.png" width="640" /></a></div>
<br /></div>
<div>
<br /></div>
<div>
Let's Extract Fields</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMthDORQkfiqMgqIFGVQLpOVK1l-VcTLn5Y_43wIF8tyI2PcD2ta_VOwrzbUlXC-wdPB7cchPMHtisUGEaL5inCOFMlZXm6_808tBYX8x87_IGdQRcRm4FEa7vyeTaXpA6KDLmWwlf4ow-/s1600/10.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="362" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMthDORQkfiqMgqIFGVQLpOVK1l-VcTLn5Y_43wIF8tyI2PcD2ta_VOwrzbUlXC-wdPB7cchPMHtisUGEaL5inCOFMlZXm6_808tBYX8x87_IGdQRcRm4FEa7vyeTaXpA6KDLmWwlf4ow-/s640/10.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br /></div>
<div>
Notice that we are extracting fields for the source type <b>dd-wrt.</b></div>
<div>
<div>
Click on one of the events.</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglgXisKuWR1GrJtM33d-ejqvKIcfKiwcBsFQ_rRJmDV0by0vPh1E4nOGIoWMU4qcZ7LqzRo9CEUKFsAjQoliMo3-1syBHv94vCb-Wjp6RKkQ8t5SJz0ttULCiZhJ234fPWP_dKyDFCYLSi/s1600/11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="272" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglgXisKuWR1GrJtM33d-ejqvKIcfKiwcBsFQ_rRJmDV0by0vPh1E4nOGIoWMU4qcZ7LqzRo9CEUKFsAjQoliMo3-1syBHv94vCb-Wjp6RKkQ8t5SJz0ttULCiZhJ234fPWP_dKyDFCYLSi/s640/11.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
</div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTTDcZ4ERneOW6AdIgVAWf00MzBML5j54JGqCWk-n8v3GwQlVwWJtUP1T3lNAd7druo5EglO28UM64svpRu0IMfCCPBqNdflR27RwOuxTZp1Z81H5DPIyIpuA3hAmaQbj3WSeuPq4s0PoN/s1600/12.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="172" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTTDcZ4ERneOW6AdIgVAWf00MzBML5j54JGqCWk-n8v3GwQlVwWJtUP1T3lNAd7druo5EglO28UM64svpRu0IMfCCPBqNdflR27RwOuxTZp1Z81H5DPIyIpuA3hAmaQbj3WSeuPq4s0PoN/s640/12.png" width="640" /></a></div>
Select <b>Regular Expression</b></div>
<div>
<b><br /></b></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhCohPW_f-_mVTxLEDPILDCIYwYIX8E28vPrwoKyqJG6l6tuS-tbxvVGYTHWzWXDi-C4olH9XAF2m7rZiJtp6DmfhaDzBz4OQWgtti0wK9cdpM9OxrBaNeCBFNnHtIvO4VT21QrzQIYxBM/s1600/13.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhCohPW_f-_mVTxLEDPILDCIYwYIX8E28vPrwoKyqJG6l6tuS-tbxvVGYTHWzWXDi-C4olH9XAF2m7rZiJtp6DmfhaDzBz4OQWgtti0wK9cdpM9OxrBaNeCBFNnHtIvO4VT21QrzQIYxBM/s640/13.png" width="640" /></a></div>
Highlight 'ACCEPT' and name it action</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQkDYbegSR0wL9vggsnXRvS3j7fof8qds6bLOS9PO1RONPj37bM4x2bWBsVi4hRXEdgrtoJb19yhyD5RUlV9OVVFpoWtvVGxD3-1Q7AzRsGOopC1fAV0krn6OOpZcvMOtkv30BbgvBst3G/s1600/14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="134" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQkDYbegSR0wL9vggsnXRvS3j7fof8qds6bLOS9PO1RONPj37bM4x2bWBsVi4hRXEdgrtoJb19yhyD5RUlV9OVVFpoWtvVGxD3-1Q7AzRsGOopC1fAV0krn6OOpZcvMOtkv30BbgvBst3G/s640/14.png" width="640" /></a></div>
<br /></div>
<div>
Now you'll see the newly extracted field</div>
<div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhplvX0_O_WXtfYGzd5OIeNBiFRkRtt3g5u1iI4LWmf9KGNiPIDAe5J2hGob_q1nzkCgpsC9lvMmVsfz93Scitg3rVTg6qHY7Jcjg-0R-1Bjjs3cbdaFtAtM55bMo8QvYl7PHUWREL3bW1S/s1600/15.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhplvX0_O_WXtfYGzd5OIeNBiFRkRtt3g5u1iI4LWmf9KGNiPIDAe5J2hGob_q1nzkCgpsC9lvMmVsfz93Scitg3rVTg6qHY7Jcjg-0R-1Bjjs3cbdaFtAtM55bMo8QvYl7PHUWREL3bW1S/s640/15.png" width="640" /></a></div>
Make sure to select permission for App</div>
<div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCfX7voUQG-1wzM5yk57IV4roHJhqsHhsYZNpnS54dVkE7Z12Xbtnc6RHF9RM1uASj1rroL5HZYv6GWYPen7r8sttmb7MlOOIDgJU1xRSyrbkjgKD3VFZwcXzvriTSXS3lCt66DfT5cabd/s1600/16.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCfX7voUQG-1wzM5yk57IV4roHJhqsHhsYZNpnS54dVkE7Z12Xbtnc6RHF9RM1uASj1rroL5HZYv6GWYPen7r8sttmb7MlOOIDgJU1xRSyrbkjgKD3VFZwcXzvriTSXS3lCt66DfT5cabd/s640/16.png" width="640" /></a></div>
Now run a search and you'll see the newly created field.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9NmKoRghtZ3Bepb93csJoqI2EkKNIDn7hlcQy3CLde0r2P2q6RmKHN195Mm1qIOpQ1q__yW29bsAZ_ILCwJAAvBvgwJM9IcO9eYh3olYQJzY4bQ-PptGPEDLJt1huHVtqYPL8ChqI_Coq/s1600/17.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="354" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9NmKoRghtZ3Bepb93csJoqI2EkKNIDn7hlcQy3CLde0r2P2q6RmKHN195Mm1qIOpQ1q__yW29bsAZ_ILCwJAAvBvgwJM9IcO9eYh3olYQJzY4bQ-PptGPEDLJt1huHVtqYPL8ChqI_Coq/s640/17.png" width="640" /></a></div>
<br />
<br />
Next step, let's add some logic to the props.conf : Go ahead and open your $SPLUNK_HOME/etc/apps/homemonitor/local/props.conf<br />
<br /></div>
<div>
You should see this entry [dd-wrt] :<br />
<br /></div>
<div>
<div style="margin: 0px;">
[dd-wrt]<br />pulldown_type = 1<br />EXTRACT-action = (?i) .*?: (?P<action>\w+)(?= )</div>
<div style="margin: 0px;">
<br /></div>
</div>
<div>
Now, let's 'normalize' the fields for Source IP, Source Port, Destination IP and Destination Port.<br />
<br />
FIELDALIAS-dst = DST as dest_ip<br />FIELDALIAS-dpt = DPT as dest_port<br />FIELDALIAS-proto = PROTO as protocol<br />FIELDALIAS-SPT = SPT as src_port<br />FIELDALIAS-SRC = SRC as src_ip<br />EVAL-direction = if(match(OUT,"eth*"), "out", "in")<br />
<br />
<div style="-webkit-text-stroke-width: 0px; font-family: Times; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px;">
</div>
<br />
<div style="-webkit-text-stroke-width: 0px; font-family: Times; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px;">
<div>
<div style="margin: 0px;">
<br /></div>
<div style="margin: 0px;">
This stanza will create a new field called direction and base the direction on what interface is being used by the field 'OUT.' If the OUT field has eth0, then it will determine the direction as 'out,' otherwise it will set the direction to 'in.' </div>
<div style="margin: 0px;">
<br /></div>
</div>
<div style="margin: 0px;">
</div>
<div>
</div>
<br />
<div style="-webkit-text-stroke-width: 0px; font-family: Times; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px;">
<div>
<div style="margin: 0px;">
<br /></div>
<div style="margin: 0px;">
Finally, we can add some lookups to enrich the data. The first lookup normalizes the action to either ACCEPT or BLOCK. The second lookup actually does reverse lookup for IP addresses. </div>
<div style="margin: 0px;">
<br /></div>
</div>
<div>
<div style="margin: 0px;">
LOOKUP-action_lookup = action_lookup action OUTPUT action2</div>
</div>
<div>
<div style="margin: 0px;">
LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host</div>
</div>
</div>
<div style="margin: 0px;">
<br /></div>
</div>
The final props.conf entry should look like this :<br />
<br />
[dd-wrt]<br />
pulldown_type = 1<br />EXTRACT-action = (?i) .*?: (?P<action>\w+)(?= )<br />FIELDALIAS-dst = DST as dest_ip<br />FIELDALIAS-dpt = DPT as dest_port<br />FIELDALIAS-proto = PROTO as protocol<br />FIELDALIAS-SPT = SPT as src_port<br />FIELDALIAS-SRC = SRC as src_ip<br />EVAL-direction = if(match(OUT,"eth*"), "out", "in")<br />LOOKUP-action_lookup = action_lookup action OUTPUT action2<br />LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host<br />
<br />
<div>
</div>
<br />
<br /></div>
<div>
I will be adding this to the default props.conf for the next release of Home Monitor 4.4.3. </div>
amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com5tag:blogger.com,1999:blog-4935085660130283180.post-13519372108899156732016-02-07T20:30:00.003-08:002016-02-07T20:31:59.386-08:00Getting Bandwidth Data into Home Monitor App version 4.4.2Super Bowl 50 is in the books and so I decided to update the Home Monitor app to include average bandwidth for your home network. I got the idea after reading the post about the<a href="https://www.reddit.com/r/technology/comments/43fi39/i_set_up_my_raspberry_pi_to_automatically_tweet" target="_blank"> Comcast customer that was tweeting </a> their complaints each time their bandwidth fell below certain rate. I decided to add this feature to the app to help show users their average Download, Upload and Ping rates.<br />
<br />
First, I downloaded the scripts necessary to get the speedtest data, found here : <a href="https://github.com/sivel/speedtest-cli">https://github.com/sivel/speedtest-cli</a><br />
<br />
Next, I tweaked the python script so that it displayed everything as a Name = Value pair, making it easy to Splunk :<br />
<br />
Download=85 Mbps<br />
Upload=60 Mbps<br />
Ping=8ms<br />
<br />
speedtest_cli.py<br />
<br />
Changed Download: to Download=, Upload: to Upload= and Ping: to Ping=<br />
<br />
Once that was done, I created the inputs for both Windows and Linux :<br />
<br />
# Bandwidth Input for Linux Machines<br />
[script://$SPLUNK_HOME/etc/apps/homemonitor/bin/speedtest.sh]<br />
disabled = true<br />
interval = 1800.0<br />
sourcetype = bandwidth_test<br />
<br />
# Bandwidth Input for Windows Machines<br />
[script://$SPLUNK_HOME\etc\apps\homemonitor\bin\speedtest_cli.py --simple]<br />
disabled = false<br />
interval = 1800.00<br />
source = bandwidth_test<br />
sourcetype = bandwidth_test<br />
<br />
Now, all you have to do is go into Settings -> Data Inputs -> Scripts and enable the proper script for your OS. I would keep it to 15 minutes (1,800 seconds) since you don't want to be running a speedtest on a high frequency which could degrade your networks performance.<br />
<br />
Thanks and enjoy,<br />
Kam<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQq-cO5m2PzbT55_iv9wtAnE4BTsxSD6MPyuvWpD51LI4r81RdNWL3AN1smEFP22iD4OgsXk_SeMhyphenhyphenZyhKIvgGOkOi7vVb1LNEuhg1htTJPbNXcFoNsEN70fE9E059zrJcrWeBh5b44LKj/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="128" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQq-cO5m2PzbT55_iv9wtAnE4BTsxSD6MPyuvWpD51LI4r81RdNWL3AN1smEFP22iD4OgsXk_SeMhyphenhyphenZyhKIvgGOkOi7vVb1LNEuhg1htTJPbNXcFoNsEN70fE9E059zrJcrWeBh5b44LKj/s320/1.png" width="320" /></a></div>
<br />amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com6tag:blogger.com,1999:blog-4935085660130283180.post-28314827877066323532015-12-26T19:25:00.004-08:002015-12-26T19:25:57.253-08:00Adding Quantam Sourcetypeprops.conf<br />
<blockquote class="tr_bq">
[syslog]<br />TRANSFORMS-changesourcetype = asus, fios, link sys, mikro, netgear, openwrt, pfsense, quantum, sophos, skyhub, tomato</blockquote>
<blockquote class="tr_bq">
[quantum]<br />FIELDALIAS-dst = DST as dest_ip<br />FIELDALIAS-dpt = DPT as dest_port<br />FIELDALIAS-proto = PROTO as protocol<br />FIELDALIAS-SPT = SPT as src_port<br />FIELDALIAS-SRC = SRC as src_ip<br />EXTRACT-action = ^[^\]\n]*\]:\s+(?P<action>\w+)<br />EVAL-direction = if(match(OUT,"eth*"), "out", "in")<br />pulldown_type = 1<br />LOOKUP-action_lookup = action_lookup action OUTPUT action2<br />LOOKUP-rdns = dnsLookup ip AS dest_ip OUTPUTNEW host as rdns_host</blockquote>
<br />
transforms.conf<br />
<br />
<blockquote class="tr_bq">
[quantum]<br /># Make sure that this matches the hostname of your router, quantum is just an example.<br />REGEX = quantum<br />SOURCE_KEY = MetaData:Host<br />FORMAT = sourcetype::quantum<br />DEST_KEY = MetaData:Sourcetype</blockquote>
<br />
For whatever reason, when I paste this into the comments bar it does not properly show the '<action>' extraction.amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com4tag:blogger.com,1999:blog-4935085660130283180.post-47727479445262426302015-10-26T17:40:00.000-07:002015-10-26T17:40:09.789-07:00Adding Splunk Stream to your home network<h2>
One Level Deeper with Splunk Stream</h2>
The data that you get from your firewall or home network device only tells you half the story and even that story is kind of boring. For the real details of your home network, you'll have to start digging into your network a little more. You can start by adding a managed switch to your environment which will allow you to span your network and collect some interesting data points. <br />
<br />
<h3>
Setting up Port Mirroring</h3>
<div>
"Port Mirroring is a method used to monitor your network traffic." Basically, your managed switch will replicate the data going through your network onto one port. You will 'tap' this port and listen to all the packets using Splunk Stream. Since most home networks are not terribly large, you can leverage a computer with not a great deal of horse power. I'm using the same hardware I used to build my pfsense firewall to build this Stream forwarder / proxy server. </div>
<div>
<br /></div>
<div>
I'm going to walk you through setting up the Cisco Switch along with the Stream Forwarder to capture this wire data. Let's start by logging into the Switch and setting up Port Mirroring:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHodpNVL3ODB6AhmHj6yrnF60oMfdUK_2oU7koud96JcJQBvabntLE6LEV0pTuuhjacUs5CvdmOEiJohy_HQP92TJlIC6VfPUx0eYCM-sJeSNPOloUTk8Ue8YEKEjWiOiHO3UdjqptzcR9/s1600/LogIn_Cisco_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="234" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHodpNVL3ODB6AhmHj6yrnF60oMfdUK_2oU7koud96JcJQBvabntLE6LEV0pTuuhjacUs5CvdmOEiJohy_HQP92TJlIC6VfPUx0eYCM-sJeSNPOloUTk8Ue8YEKEjWiOiHO3UdjqptzcR9/s640/LogIn_Cisco_1.png" width="640" /></a></div>
<div>
<br /></div>
<div>
First, add the ports you want to mirror.<br />
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8-PS_LxJatKZMwd-zJ800bFRl9oS0HoUa1CTbp6rK1NYZMFQfcJDgaI9IiGKfx092ZwueCgFZ2EldiGZY2t69gmY1NC9zERhOxnF3JGEo-Vjqq5smsLkGn3_gpSgdCAm1TzTmWgmkTO4L/s1600/Cisco_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8-PS_LxJatKZMwd-zJ800bFRl9oS0HoUa1CTbp6rK1NYZMFQfcJDgaI9IiGKfx092ZwueCgFZ2EldiGZY2t69gmY1NC9zERhOxnF3JGEo-Vjqq5smsLkGn3_gpSgdCAm1TzTmWgmkTO4L/s640/Cisco_2.png" width="640" /></a></div>
<div>
<br />
Step 1 - Enable the Ports<br />
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimJTkhLodldx1mxWYWSCC1Mg09GOdZ8qIFgBrt_f5oXQL9WZN2m1OC7u_Axx-3-oAIO3iylgLyaanPmKjGk1OCqV6T0XJsf4vtFzNfPWAdepuk8JeoZWPSU-PEHF9RB03B3ylh5Vbsuv2f/s1600/Cisco_3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimJTkhLodldx1mxWYWSCC1Mg09GOdZ8qIFgBrt_f5oXQL9WZN2m1OC7u_Axx-3-oAIO3iylgLyaanPmKjGk1OCqV6T0XJsf4vtFzNfPWAdepuk8JeoZWPSU-PEHF9RB03B3ylh5Vbsuv2f/s640/Cisco_3.png" width="640" /></a></div>
<br />
Step 2 - Enable the mirrored ports and the Admin Port. In this example, I've mirrored ports G1-G7 onto port G8. I will plug my Splunk Stream Forwarder to this port (G8).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyeR5vyVS16xINthkGYFjm_RpfSETtaKpMtHoytnpoDMNBdJ7rwJfD3Pfh31kHbav-1zQtdJT6J1UqVOq6yQNCFwBNWTMocat-wN-3R5q-JdH7b4AT8TDitcERt6iDdRb-dJAiWeUziJQ7/s1600/Cisco_4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyeR5vyVS16xINthkGYFjm_RpfSETtaKpMtHoytnpoDMNBdJ7rwJfD3Pfh31kHbav-1zQtdJT6J1UqVOq6yQNCFwBNWTMocat-wN-3R5q-JdH7b4AT8TDitcERt6iDdRb-dJAiWeUziJQ7/s640/Cisco_4.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3>
Setup Splunk Stream Forwarder</h3>
</div>
<div>
Step 1 - Download <a href="https://splunkbase.splunk.com/app/1809/" target="_blank">Splunk Stream</a></div>
<div>
<br /></div>
<div>
Step 2 - Setup Spunk Stream</div>
<div>
<br /></div>
<div>
I will not go deeply into this setup since it is well documented on the <a href="http://docs.splunk.com/Documentation/StreamApp/latest/DeployStreamApp/InstallSplunkAppforStream" target="_blank">Splunk Stream Docs</a> page. (http://docs.splunk.com/Documentation/StreamApp/latest/DeployStreamApp/InstallSplunkAppforStream) </div>
<div>
<br /></div>
<div>
Once you've setup your Splunk Stream Server, you should go to your machine's Stream page. For example, if your stream server's hostname is "stream," simply go to https://stream:8889 and you should see this page:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitKTlTIA5Jow6O2bPEkKth4y7f-nBZB6xi-Xk2auu3UJRsV49o_IQf-iIKsRVMCl8suIYhZ7UdsPspguMoxYY8iAB_F-50Z_IVnw82SjUXqjUyK8l3QYLd5zZaHfdrc6OYpf-U0NADG16-/s1600/stream.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="256" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitKTlTIA5Jow6O2bPEkKth4y7f-nBZB6xi-Xk2auu3UJRsV49o_IQf-iIKsRVMCl8suIYhZ7UdsPspguMoxYY8iAB_F-50Z_IVnw82SjUXqjUyK8l3QYLd5zZaHfdrc6OYpf-U0NADG16-/s640/stream.png" width="640" /></a></div>
<div>
<br /></div>
<div>
This will show you that you are collecting packets from your 'SPAN port' and that you should have data in your Splunk indexer. Run this simple search command to see if you are collecting data :</div>
<div>
<br /></div>
<blockquote class="tr_bq">
index=main sourcetype=stream* | stats count by sourcetype</blockquote>
<div>
<br /></div>
<div>
This will show you if you are receiving data from your Splunk Stream Forwarder. </div>
<div>
<br /></div>
<div>
Hope that helps and as always happy Splunking! </div>
<h4>
Suggested Parts List</h4>
<div>
Here's what I bought to make all of this possible :</div>
<div>
<br /></div>
<div>
<a href="http://amzn.com/B004OA721C" target="_blank">Cisco SG200-08 8-port Gigabit Smart Switch (SLM2008T-NA)</a> ~ $90</div>
<div>
<br /></div>
<div>
If you don't have a dual NIC server, then I would suggest you build one like this which I used to build my pfsense firewall. </div>
<div>
<br /></div>
<div>
<a href="http://amiracle19.blogspot.com/2014/07/pfsense-by-passing-fios-and-comcast.html" target="_blank">Splunk Stream Forwarder / Proxy Server</a> </div>
<div>
<br /></div>
<div>
<br /></div>
amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com2tag:blogger.com,1999:blog-4935085660130283180.post-36412690559831862492015-09-20T14:26:00.001-07:002015-09-20T14:26:11.876-07:00Troubleshooting home | monitor > app<h2>
Setting Up Splunk</h2>
<h3>
Linux / Mac OSX Users</h3>
<div>
Let's start by setting up your Splunk instance on a *NIX system. First, install the Splunk binaries (.dmg, .rpm, .deb, or .tgz) and start your Splunk instance. If you want Splunk to start on reboots, just run the boot-start -user splunk as root for your instance:</div>
<div>
<br /></div>
<div>
my-host$ sudo opt/splunk/bin/splunk start enable boot-start -user splunk</div>
<h4>
Firewalls</h4>
<div>
If you have iptables enabled, make sure to open the ports for syslog in-bound (UDP 514) and out-bound TCP 8000, 8089. If you are going to use Splunk Stream enable TCP port 8889. Lastly, if you're going to enable forwarding, you should open TCP port 9997. </div>
<div>
<br /></div>
<h3>
Windows Users</h3>
<div>
Create a firewall rule and include the following ports:</div>
<div>
<br /></div>
<div>
In-bound</div>
<div>
UDP port 514</div>
<div>
<br /></div>
<div>
Out-bound</div>
<div>
TCP port 8089, 8000, 8889, 9997</div>
<h2>
Getting Data In</h2>
<h3>
Syslog</h3>
<div>
First, let's make sure your router / device can send data in via syslog. Most devices send data over UDP port 514, which is the default syslog. On your device, you will to set the IP Address of your Splunk server as the recipient of the syslog data. </div>
<div>
<br /></div>
<div>
On your Splunk server, you will see that the UDP port 514 has already been enabled but you might have trouble collecting the data. On Linux, you have to be running as root in order to listen to port 514. If you do not want to run as root, you will need to modify the port on which syslog is sending data. </div>
<div>
<br />
(http://answers.splunk.com/answers/310324/home-monitor-how-to-configure-a-netcomm-modem-rout.html#answer-310832)<br />
<br /></div>
<div>
If you're modem allows you to change the port from 514 to 1514 (UDP), then all you have to do is modify the input in Splunk to reflect the new UDP port.</div>
<div>
<br /></div>
<div>
You can also just move the inputs.conf file from the $SPLUNK_HOME/etc/apps/homemonitor/default/inputs.conf and put it into $SPLUNK_HOME/etc/apps/homemonitor/local and just make the following changes:</div>
<div>
<br /></div>
<div>
[udp://514]</div>
<div>
connection_host = dns</div>
<div>
sourcetype=syslog</div>
<div>
index = homemonitor</div>
<div>
disabled = 0</div>
<div>
</div>
<div>
to</div>
<div>
<br /></div>
<div>
[udp://1514]</div>
<div>
connection_host = dns</div>
<div>
sourcetype=syslog</div>
<div>
index = homemonitor</div>
<div>
disabled = 0<br />
<br /></div>
<div>
</div>
<div>
To do this from the Web GUI, go to Settings -> Data Inputs -> UDP and click on Add. For the port put in 1514 and then click Next.</div>
<div>
<br /></div>
<div>
In the next screen, under Index, make sure you select the dropdown and select Homemonitor. Click Review and then click Submit.</div>
<div>
<br /></div>
<div>
That should get you to start collecting data into your home monitor app. To check, go to the Search in the homeomitor app and type in index=homemonitor and hit enter. You should start to see data streaming into Splunk.</div>
<h3>
Data is in, but no dashboards are populating?!</h3>
<div>
First step in troubleshooting is to make sure that the data is flowing into the app. Let's start by running a simple search. Open the home | monitor > app and click on Search. Now run the following search :</div>
<div>
<br /></div>
<blockquote class="tr_bq">
index=homemonitor | stats count by sourcetype</blockquote>
<div>
<br /></div>
<div>
If you do not see any events, then the data is not coming in and you should double check 1) the Splunk IP on your device 2) Firewall rules and that UDP 514 is allowed</div>
<div>
<br /></div>
<div>
OK, I got events, but they are all just coming in as syslog! </div>
<div>
<br /></div>
<div>
What is happening is that the data is coming in as syslog but is not being transformed into the source type of your device. You have some options to fix this.</div>
<div>
<br /></div>
<div>
<b>Option 1: Hard Code the source type on the input:</b></div>
<div>
<br /></div>
<div>
Via the Web UI click on Settings -> Data Inputs -> UDP -> 514 . From here, it will open that port and you can change the Manual setting of syslog to "From list" and then select your device type (e.g. asus, linksys, fios, pfsense, etc.). Click Save. </div>
<div>
<br /></div>
<div>
From the command line (CLI) edit the $SPLUNK_HOME/etc/apps/homemonitor/local/inputs.conf </div>
<div>
<div>
[udp://514]</div>
<div>
connection_host = dns</div>
<div>
sourcetype=syslog</div>
<div>
index = homemonitor</div>
<div>
disabled = 0<br />
<br />
Change "syslog" to your corresponding device type.<br />
<br />
Once it's been saved then you will start receiving the data as the source type of your device.<br />
<br /></div>
</div>
<div>
<b>Option 2: Change the hostname to match the modem type:</b></div>
<div>
<br /></div>
<div>
Go into your device and change the hostname from the default to just the vendor name of your modem. For example, if you have an Asus router, just make the hostname "asus". This will allow Splunk to automatically set the source type to asus via the transforms.conf file. </div>
<div>
<br /></div>
<div>
Hopefully this should cover most of the problems that you'll encounter setting up the Splunk app for home | monitor >. I've been meaning to setup this page for some time now. I will be posting a new video shortly going over setting up the new 4.x app with Splunk 6.x.</div>
amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com35tag:blogger.com,1999:blog-4935085660130283180.post-7431960147059367092015-09-20T11:04:00.001-07:002015-09-20T11:05:54.279-07:00Adding OpenWRT sourcetypeI've made some modifications to the props.conf and added the new source type:<br />
<br />
<blockquote class="tr_bq">
[syslog]<br />
TRANSFORMS-changesourcetype = fios, pfsense, asus, netgear, skyhub, linksys, mikro, openwrt<br />
[openwrt]<br />
# Based on Asus RT-N66U router syslog output.<br />
FIELDALIAS-dst = DST as dest_ip<br />
FIELDALIAS-dpt = DPT as dest_port<br />
FIELDALIAS-proto = PROTO as protocol<br />
FIELDALIAS-SPT = SPT as src_port<br />
FIELDALIAS-SRC = SRC as src_ip<br />
EXTRACT-action = ^[^\]\n]*\]\s+(?P<action>\w+)<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><br /></span>pulldown_type = 1<br />
LOOKUP-action_lookup = action_lookup action OUTPUT action2</blockquote>
<br />
And made a quick change to the transforms.conf to include openwrt :<br />
<br />
<blockquote class="tr_bq">
[openwrt]<br />
# Make sure that this matches the hostname of your router, openwrt is just an example.<br />
REGEX = openwrt<br />
SOURCE_KEY = MetaData:Host<br />
FORMAT = sourcetype::openwrt<br />
DEST_KEY = MetaData:Sourcetype</blockquote>
<br />
<br />
Special thanks to @LodiHensen [twitter] for helping test out this source type on OpenWRT. <br />
<br />
I will add these updates to the next release of the home | monitor > app, but for now you can copy these entires for your props.conf and transforms.conf files.<br />
<br />amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com1tag:blogger.com,1999:blog-4935085660130283180.post-44056966284963234582015-06-16T16:38:00.003-07:002015-06-16T16:46:27.349-07:00Sophos Sourcetype Added<div class="p1">
Here are the configuration changes you'll need to make to add Sophos firewalls to the home | monitor > 4.0 . Please note that the direction field does not exist, so some of the pfsense dashboards will not fully populate.</div>
<div class="p1">
<br /></div>
<div class="p1">
transforms.conf </div>
<br />
[sophos]<br />
REGEX = sophos<br />
SOURCE_KEY = MetaData:Host<br />
FORMAT = sourcetype::sophos<br />
DEST_KEY = MetaData:Sourcetype<br />
<br />
<br />
<div>
<div class="p2">
props.conf</div>
<br />
[sophos]</div>
<div>
FIELDALIAS-srcip = srcip as src_ip</div>
<div>
FIELDALIAS-srcport = srcport as src_port</div>
<div>
FIELDALIAS-dstip = dstip as dest_ip</div>
<div>
FIELDALIAS-dstport = dstport as dest_port</div>
<div>
FIELDALIAS-dstmac = dstmac as dest_mac</div>
<div>
FIELDALIAS-proto = proto as protocol</div>
<div>
FIELDALIAS-fwrule = fwrule as firewall_rule<br />
<br />
action_lookups.csv<br />
<br />
drop, BLOCK<br />
accept,ACCEPT <br />
<br />
After you add these entries, make sure to restart your Splunk instance. I'll update the default conf and csv files for a later release (4.0.2). <br />
<br />
<br /></div>
amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com0tag:blogger.com,1999:blog-4935085660130283180.post-70017777801711406692015-05-21T22:04:00.002-07:002015-05-21T22:08:23.294-07:00New home | monitor > 4.0 ReleasedI just finished working on home | monitor > 4.0 with new enhancements which includes:<br />
<br />
Splunk Stream Support - used to power new D3 dashboards<br />
<br />
New D3 visualizations<br />
<br />
Sankey<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPca0sMbi2XS1Dnwsef4NzL1YAI4CvEGqdP7b-sulfhm2y-t3Wmct1v67LWg8kLOOQqJRhp0fFpnsZTf67QJJfUBfulyccJ2VSdCq8Q4bibX2_aoFdsVSq5_QCDYRAVv8-F9yyVFVeLNwR/s1600/Screen+Shot+2015-05-22+at+12.39.56+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="157" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPca0sMbi2XS1Dnwsef4NzL1YAI4CvEGqdP7b-sulfhm2y-t3Wmct1v67LWg8kLOOQqJRhp0fFpnsZTf67QJJfUBfulyccJ2VSdCq8Q4bibX2_aoFdsVSq5_QCDYRAVv8-F9yyVFVeLNwR/s400/Screen+Shot+2015-05-22+at+12.39.56+AM.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Tag Cloud</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcY2jPFB3_a3qufMqRmfBeOamM3AZPGDf0IiUPSKxVnYuNq1pYWqep6WnM2q5fYgcW9gpr2ldNFinCt5AeyY7HgCPdltuqtzqfFFh2r7TpruAbG88AYKWvYvB8CHgFxZ9PhupNHgjtYJJg/s1600/Screen+Shot+2015-05-22+at+12.39.20+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="212" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcY2jPFB3_a3qufMqRmfBeOamM3AZPGDf0IiUPSKxVnYuNq1pYWqep6WnM2q5fYgcW9gpr2ldNFinCt5AeyY7HgCPdltuqtzqfFFh2r7TpruAbG88AYKWvYvB8CHgFxZ9PhupNHgjtYJJg/s400/Screen+Shot+2015-05-22+at+12.39.20+AM.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
Expanded workflow for both pfSense and FiOS routers<br />
<br />
Third Party lookup using MXToolbox.com<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfCNJYCnX3YPbPQsCAdx2lRm-zhe_glX0p1HA0lsTS-tGOVycTKLc9E6nfLcrEwZTVif_BxCPkhAKoZpBzsozJfUMJapRdKl6URLIRp2NQmJafw9IlG7swNMFmpalNsGZu1KMmXR89vH9q/s1600/Screen+Shot+2015-05-22+at+1.07.00+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="210" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfCNJYCnX3YPbPQsCAdx2lRm-zhe_glX0p1HA0lsTS-tGOVycTKLc9E6nfLcrEwZTVif_BxCPkhAKoZpBzsozJfUMJapRdKl6URLIRp2NQmJafw9IlG7swNMFmpalNsGZu1KMmXR89vH9q/s400/Screen+Shot+2015-05-22+at+1.07.00+AM.png" width="400" /></a></div>
<br />
New home | monitor > logo<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhecB00qIfPWr5EmpBdUASSuRdu1FsfcEQfvlWsH2F1x8AMc5C_CunUjI9sxsHJ2dkIyU6c11vHqbnBQS8LfVZDQF2Ls7RFFNpHQkRBBvhO8Zg2kvP3rfGEmsei_bBn0bDfYPlnTNu2XGUa/s1600/appLogo_2x.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhecB00qIfPWr5EmpBdUASSuRdu1FsfcEQfvlWsH2F1x8AMc5C_CunUjI9sxsHJ2dkIyU6c11vHqbnBQS8LfVZDQF2Ls7RFFNpHQkRBBvhO8Zg2kvP3rfGEmsei_bBn0bDfYPlnTNu2XGUa/s1600/appLogo_2x.png" /></a></div>
<br />
Updated searches and dashboards so that they all work and are CIM 4.2 compliant.<br />
<br />
Enjoy the updates, I will be posting a video on how to use the app soon.<br />
<br />
Thanks,<br />
Kam<br />
<br />
<br />amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com14tag:blogger.com,1999:blog-4935085660130283180.post-15559735220045176712015-04-10T19:48:00.001-07:002015-04-10T20:02:17.359-07:00Home Monitor 3.2.1 ReleasedJust released Home Monitor 3.2.1!<br />
<br />
This version now supports the latest version of pfSense (2.2.1) with the latest logging settings. Here is the documentation for the latest logging format for pfSense: <a href="https://doc.pfsense.org/index.php/Filter_Log_Format_for_pfSense_2.2">https://doc.pfsense.org/index.php/Filter_Log_Format_for_pfSense_2.2</a><br />
<br />
This new version of Home Monitor App 3.2.1 also includes a basic workflow that allows users to walk through the dynamic dashboards. You can start with the Home Network Device List and see which devices have registered on your network. This host list is generated by devices receiving IP addresses from your home router. Click on any of the IP's and the dashboard below will populate with the external connections the device has been connecting. Click on the "Destination_IP" and this will take you to the "Detective Dashboard" that shows you what other internal IP's the server has been talking to. Finally, click on the "dest_ip" in the panel labeled "External IP's Attempting Connections inbound," and you'll see additional information about that destination ip.<br />
<br />
There are other workflows built into the app and I will cover them on an online video later this month. For now, enjoy the enhancements and please let me know any fixes or issues you run into with this new build.<br />
<br />
Thanks!<br />
Kamamiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com1tag:blogger.com,1999:blog-4935085660130283180.post-82852433346956574082015-02-25T03:19:00.005-08:002015-02-25T09:54:26.124-08:00Mikro Tik firewall source type<br />
Here's the changes that need to make for you to add a Mikro Tik firewall to the Home Monitor App.<br />
<div>
<br /></div>
<div>
First, edit the <b>props.conf</b> in your $SPLUNK_HOME/etc/apps/homemonitor/local directory:</div>
<div>
<blockquote class="tr_bq">
<div style="font-size: 16px;">
[syslog]</div>
<div style="font-size: 16px;">
TRANSFORMS-changesourcetype=fios, pfsense, asus, netgear, skyhub, linksys, mikro</div>
</blockquote>
</div>
<div>
<blockquote class="tr_bq">
<div style="font-size: 16px;">
[mikro]</div>
<div style="font-size: 16px;">
EXTRACT-hostname = ^(?:[^ \n]* ){7}(?P<hostname>\w+)</div>
<div style="font-size: 16px;">
EXTRACT-transport = ^(?:[^,\n]*,){2}\s+\w+\s+(?P<transport>\w+)</div>
<div style="font-size: 16px;">
EXTRACT-src_ip,src_port = ^(?:[^,\n]*,){4}\s+(?P<src_ip>[^:]+)[^:\n]*:(?P<src_port>\d+)</div>
<div style="font-size: 16px;">
EXTRACT-dest_ip,dest_port = ^[^>\n]*>(?P<dest_ip>[^:]+)[^:\n]*:(?P<dest_port>\d+)</div>
<div style="font-size: 16px;">
EXTRACT-process = ^(?:[^ \n]* ){9}(?P<process>\w+)</div>
<br />
<div style="font-size: 16px;">
EXTRACT-nat_ip,nat_port = ^(?:[^>\n]*>){2}(?P<nat_ip>[^:]+)[^:\n]*:(?P<nat_port>[^\)]+)</div>
</blockquote>
Next, edit the <b>transforms.conf</b> to include the mikro source type:<br />
<br />
<blockquote class="tr_bq">
[mikro]<br />
# Make sure that this matches the hostname of your router, mikro is just an example.<br />
# Replace the field below with your router / firewall / modem's hostname.<br />
REGEX = mikro<br />
SOURCE_KEY = MetaData:Host<br />
FORMAT = sourcetype::mikro<br />
DEST_KEY = MetaData:Sourcetype</blockquote>
Please note that this does not have blocks or accepts in the logs, so I cannot populate the blocked or accepted dashboards. <br />
<br />
Thanks and enjoy!<br />
<br />
-Kam</div>
amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com0tag:blogger.com,1999:blog-4935085660130283180.post-84492732050249555482015-02-10T07:46:00.001-08:002015-02-10T07:46:54.180-08:00Juniper Source Type<div style="font-family: Calibri, sans-serif; font-size: 14px;">
The documentation is a combination from the online docs.splunk.com with some of the answers found on answers.splunk.com. In this situation, we need to make an entry into the props.conf located ($SPLUNK_HOME/etc/apps/homemonitor/local/props.conf). The plan is to create ‘field-aliases’ which will convert the way juniper logs the events to CIM (Common Information Model). Here’s how I would make the changes:</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">
<br /></div>
<blockquote class="tr_bq">
[juniper]<br />
FIELDALIAS-source-address = source-addresss as src_ip<br />
FIELDALIAS-source-port = source-port as src_port<br />
FIELDALIAS-destintation-address = destintation-addresss as dest_ip<br />
FIELDALIAS-destination-port = destination-port as dest_port</blockquote>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">
<br /></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">
Then you’ll need to make an entry into the transforms.conf located ($SPLUNK_HOME/etc/apps/homemonitor/local/transforms.conf) to add the juniper transformation from syslog to juniper sourcetype:</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">
<br /></div>
<blockquote class="tr_bq">
[juniper]<br />
REGEX = juniper<br />
SOURCE_KEY = MetaData:Host<br />
FORMAT = sourcetype::juniper<br />
DEST_KEY = MetaData:Sourcetype</blockquote>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">
<br /></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">
Once you’ve added this entry, just make sure that your hostname of your firewall matches the “REGEX = juniper” entry. I’m assuming that the hostname of the firewall is juniper. </div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">
<br /></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">
Lastly, we’ll need to build some lookups to help this along. I’ll need a little more data to see how the juniper firewall categorizes blocks and accepts, if it uses trust / untrust. Also, I’ll need to determine what the protocol-id translates to as well, (does 6 = TCP?); again we’ll need another lookup for that. All this can be done rather easily, we just need to better understand the data coming from the juniper firewall. </div>
<div>
<br /></div>
amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com0tag:blogger.com,1999:blog-4935085660130283180.post-84867255923829993822015-02-09T09:19:00.000-08:002015-02-09T09:20:58.905-08:00Adding the Linksys sourcetypeHow to add the Linksys router into your Home Monitor App. <br />
<br />
First think you'll need to do is modify the <b>props.conf</b> in your $SPLUNK_HOME/etc/apps/homemonitor/local/ directory. Here is the entry you should use:<br />
<blockquote class="tr_bq">
<span style="background-color: #00e6e5; font-variant-ligatures: no-common-ligatures;">[</span>linksys<span style="background-color: #00e6e5; font-variant-ligatures: no-common-ligatures;">]</span>DATETIME_CONFIG = CURRENT<br />NO_BINARY_CHECK = true<br />SHOULD_LINEMERGE = false<br />category = Custom<br />pulldown_type = true<br />EXTRACT-src_ip,dest_ip,linksys_src_port,action = ^(?P<src_ip>[^ ]+) to (?P<dest_ip>[^:]+):(?P<linksys_src_port>[a-z]+) is (?P<action>.+)<br />LOOKUP-linksys_src_port_lookup = linksys_src_port_lookup linksys_src_port OUTPUTNEW src_port</blockquote>
<br />
<span style="font-family: Times; font-size: small;">Once you've modified that file, then you'll need to make another change to your <b>transforms.conf</b> in the same location</span><span style="font-family: inherit;"> $SPLUNK_HOME/etc/apps/homemonitor/local/ . Here is the entry you'll need to make:</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
[linksys]<br /># Make sure that this matches the hostname of your router, linksys is just an example.<br />REGEX = linksys<br />SOURCE_KEY = MetaData:Host<br />FORMAT = sourcetype::linksys<br />DEST_KEY = MetaData:Sourcetype</blockquote>
</blockquote>
This will require that either you change the line REGEX = linksys to REGEX = your router's hostname. <br />
<br />
Lastly, you'll need to make some lookup files that will help get the dashboards populated with data from linksys routers. <br />
<br />
First, modify the existing lookup file "action_lookup.csv" and add the following to the end of that file:<br />
<br />
<blockquote class="tr_bq">
blocked, BLOCK<br />
accepted, ACCEPT</blockquote>
Here's the last lookup file you'll need to populate the dashboards. This basically changes the protocol (https, http, etc.) to src_port (443, 80..). Here's the lookup file (linksys_src_port.csv) :<br />
<br />
<blockquote class="tr_bq">
linksys_src_port, src_port<br />
https,443<br />
http,80<br />
ssh,22<br />
smtp,25<br />
pop,110</blockquote>
<br />
Once you've added it, you can put that into your app and make it into an automatic lookup.<br />
<br />
Enjoy,<br />
Kamamiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com0tag:blogger.com,1999:blog-4935085660130283180.post-60191522041174173032015-02-08T08:42:00.003-08:002015-02-08T08:57:30.295-08:00Home Monitor V.3.1.0 for Splunk ReleasedI've finished the latest version of the <a href="https://apps.splunk.com/app/1214/" target="_blank">Home Monitor App 3.1.0</a> (<a href="https://apps.splunk.com/app/1214/">https://apps.splunk.com/app/1214/</a>) and it now supports the Asus router as well as pfSense firewalls. I have not tested the functionality with Netgear or Skyhub routers, please let me know how they work and if they need any modifications to the app. I have not tested OpenWRT either, but please feel free to let me know if any changes need to be made. <br />
<br />
I've also updated my GitHub Repo : <a href="https://github.com/amiracle/homemonitor.git">https://github.com/amiracle/homemonitor.git</a> where you can feel free to look at the components and make any changes to the app. <br />
<br />
So, what's new?<br />
<br />
<b>Source type by hostname or manual</b><br />
<br />
This version of the app can either rely on your router's hostname to configure the sourcetype, or you can select it manually on the Data Inputs page. For example, if you have a fios router, and the hostname is fios, then the props.conf and transforms.conf will work together to change the sourcetype to fios. (The reason I did this was that it helepd during my testing having Splunk automatically pickup and change the sourcetype on the fly for me.)<br />
<br />
Once the data input is in (more on that below), you will be able to see all of the dashboards populate with your data. I even normalized the fields and the output of some of the fields using a lookup. This allows my Asus router and my pfSense firewall to have the same output as my FiOS router. You'll see that there are two fields, 'action' and 'action2' in the interesting fields. The lookup, named action_lookup.csv, will convert the action to a normalized BLOCK or ACCEPT instead of DROP or pass. This allows all the dashboards to populate regardless of your router. There are some dashboards that WILL NOT populate since they have FiOS specific fields in the search.<br />
<br />
<b>How it works:</b><br />
<br />
The props.conf has the following entry:<br />
<br />
<blockquote class="tr_bq">
[syslog]<br />
TRANSFORMS-changesourcetype=fios, pfsense, asus, netgear, skyhub</blockquote>
<br />
Then the transforms.conf file takes the source type and changes it depending on the hostname of your router:<br />
<br />
<blockquote class="tr_bq">
[fios]<br />
# Make sure that this matches the hostname of your router, fios is just an example.<br />
<b>REGEX = fios</b>SOURCE_KEY = MetaData:Host<br />
FORMAT = sourcetype::fios<br />
DEST_KEY = MetaData:Sourcetype</blockquote>
<br />
The key here is that your router's hostname either starts with or contains 'fios' in order for the change to occur automatically. Otherwise, you'll see your data come in as syslog and it will NOT have any of the proper field extractions. <br />
<br />
<b>You can also manually change the sourcetype to fios, asus, pfsense, netgear or skyhub.</b><br />
<br />
<b>Normalized Data : </b><br />
<br />
In order to get the dashboards to populate regardless of router, I had to normalize the fields from all the routers. Specifically, the 'action' field which told me if the traffic was 'BLOCKED' or 'ACCEPTED.' Since the Asus router used 'DROP' instead of BLOCKED and the pfSense firewall used 'block' instead of BLOCKED', I had to use a lookup and create a new field, 'action2.'<br />
<br />
With this in place, I was able to populate all of my dashboards with blocks and accepts regardless of the router I used. <br />
<br />
<b>Updated Dashboards : </b><br />
New updated dashboards - I've gone through and vetted all the dashboards to make sure they make some logical sense. I stopped using the 'process' field since it did not exist in all the routers syslog data. Instead, I determined that outbound connections were iniated by src_ip = 192.168.* and inbound connections were iniated by NOT src_ip=192.168.* .<br />
<br />
<b>Future Updates: </b><br />
<br />
I was thinking about creating a setup page, so more advanced users can configure the app to suite their customized networks. When I get more motivation, I'll work on setting up tags for local networks and a setup page that allows you to change some of the inputs or specify your local network IP address space.amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com8tag:blogger.com,1999:blog-4935085660130283180.post-66546193468845200952014-10-15T10:01:00.000-07:002015-01-21T18:52:07.117-08:00Home Monitor V.3.0.2 for Splunk ReleasedAfter an inspiring .conf 2014, I finally decided to push out the latest version of my Home Monitor app for Splunk, version 3.0.4 maps. <br />
<br />
Additionally, I cleaned up many of the dashboards and made them a little more interactive. They now have time pickers and even an interactive form to track down IP Addresses that are trying to gain unauthorized access to your FiOS router. <br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5xaKxgEidhQTVPtlhEXSj_BT1Vmpwu7qiY-HLoMyRKLeWrYsvdZ_BWCaatuQd8Sq0bU3YcKHI8N7b0FsOB5-hAVtuPDk9b1qU-05NSphrVFdur914aaThTllDpXlCxPu_7ICd-pdGQiuA/s1600/bad_guy.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5xaKxgEidhQTVPtlhEXSj_BT1Vmpwu7qiY-HLoMyRKLeWrYsvdZ_BWCaatuQd8Sq0bU3YcKHI8N7b0FsOB5-hAVtuPDk9b1qU-05NSphrVFdur914aaThTllDpXlCxPu_7ICd-pdGQiuA/s1600/bad_guy.png" height="165" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">New Interactive 'Bad Guys' Dashboard</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgCz3H9RfOBm5ILb7l75o9bS2JDTRDZQqgJ6lQ0NdtsDhiZ_wLOSAV4aXlfrXNlkHnYZRE6nVIYg6Zr_Tp2YumbgWACykZZLaKBXSs6iMzHBIbCR7WhC8mJhrzjjNyMII04Hj3ph3prcsk/s1600/traffic_trends.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgCz3H9RfOBm5ILb7l75o9bS2JDTRDZQqgJ6lQ0NdtsDhiZ_wLOSAV4aXlfrXNlkHnYZRE6nVIYg6Zr_Tp2YumbgWACykZZLaKBXSs6iMzHBIbCR7WhC8mJhrzjjNyMII04Hj3ph3prcsk/s1600/traffic_trends.png" height="179" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">New Traffic Trends Dashboard</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie8rKal4N8Oxc-b9KyuzQWJ0-JfApcSb1N-yn-rLnGiQo4q5n5BHasKpdcNfShIYB3_rBkFqYx8NiYuwDaEFEHL8fQs4422IkPVCnhjkCs90PVKViSqShIJ5cTearF3Nq_49EIhu8zu2yr/s1600/maps.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie8rKal4N8Oxc-b9KyuzQWJ0-JfApcSb1N-yn-rLnGiQo4q5n5BHasKpdcNfShIYB3_rBkFqYx8NiYuwDaEFEHL8fQs4422IkPVCnhjkCs90PVKViSqShIJ5cTearF3Nq_49EIhu8zu2yr/s1600/maps.png" height="185" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Enhanced Maps Dashboard</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEior-ns-5Ds2BE28ZVO-x30cvoki8A02HbnVFwx8nut5oI3K52t1u9FGzVIiqqu6eOEIdMLDQxS6t1CoCRSgN_W6-gTdiwZ2belyFS_7deOIMivNEDr5jHAnVLtB62Ee_Nz4lsFNRrJFUWf/s1600/traffic_flows.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEior-ns-5Ds2BE28ZVO-x30cvoki8A02HbnVFwx8nut5oI3K52t1u9FGzVIiqqu6eOEIdMLDQxS6t1CoCRSgN_W6-gTdiwZ2belyFS_7deOIMivNEDr5jHAnVLtB62Ee_Nz4lsFNRrJFUWf/s1600/traffic_flows.png" height="173" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">New Traffic Flows Dashboard</td></tr>
</tbody></table>
<br />
<br />
Lastly, I made some changes on the back end on how the data comes in and gets indexed. Here are the technical details on what I did, if you're not interested you can stop reading now.<br />
<br />
My dilemma was that I'm collecting data from both my pfsense firewall and my parent's FiOS router using a Raspberry Pi syslog server which forwards the data to my Splunk instance at home. (That can be another post if you're interested on how I pulled that off.) Since all my data was coming into Splunk as syslog, I needed a way to 'split' the data into two different source types, fios and pfsense. <br />
<br />
Here is how I accomplished this task.<br />
<br />
First, here is the inputs.conf (copy the sample file in the default directory and move it to the local directory)<br />
<br />
inputs.conf<br />
<br />
[udp://514]<br />
connection_host = dns<br />
sourcetype = <strike>syslog </strike><span style="background-color: yellow;">fios</span><br />
index = homemonitor<br />
disabled = 0<br />
<br />
<strike>Next, I used the transforms.conf and the props.conf to rename the source type accordingly.</strike><br />
<strike><br /></strike>
<strike>props.conf</strike><br />
<strike>[syslog]</strike><br />
<strike>TRANSFORMS-changesourcetype = fios, pfsense</strike><br />
<strike><br /></strike>
<strike>transforms.conf</strike><br />
<strike>[fios]</strike><br />
<strike>REGEX = verizon.net</strike><br />
<strike>SOURCE_KEY = MetaData:Host</strike><br />
<strike>FORMAT = sourcetype::fios</strike><br />
<strike>DEST_KEY = MetaData:Sourcetype</strike><br />
<strike><br /></strike>
<strike>[pfsense]</strike><br />
<strike>REGEX = firewall.home.com</strike><br />
<strike>SOURCE_KEY = MetaData:Host</strike><br />
<strike>FORMAT = sourcetype::pfsense</strike><br />
<strike>DEST_KEY = MetaData:Sourcetype</strike><br />
<br />
<strike>Let me explain the work flow. First, the inputs.conf file identifies the data coming into port 514 as syslog. Then, the props.conf file finds the syslog sourcetype and will send it over to the transforms.conf to change the source type from syslog to either pfsense or fios depending on the matching criteria, in my case the REGEX for the host. Since the fios data comes in from *.verizon.net, I can use verizon.net to trigger the match. Similarly, my pfsense firewall has a hostname which resolves that I use to match with and convert the sourcetype from syslog to pfsense.</strike><br />
<span style="background-color: yellow;">(I'm not ready to push this out to production just yet, so I made the fix in 3.0.4 that fixes the dashboards for all the FiOS users.)</span><br />
<br />
With all that done, now I can switch between both source types and view the charts and graphs for my home network as well as my parent's without having to make any major changes to my configuration files. It's a simple drop down that I have on my dashboards (which I have not included in the v.3.0.4 release but might in a later release depending on demand.)<br />
<br />
Thanks and enjoy!<br />
<br />
-Kamamiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com6tag:blogger.com,1999:blog-4935085660130283180.post-50441536329523419332014-07-14T11:07:00.002-07:002014-07-14T11:08:12.628-07:00pfSense by-passing FiOS and Comcast hardware<b>Taking my home network to the next level...</b><br />
<br />
First of all, thanks for continuing to read my blog posts. I've finally had some time to put together a post on what I've done with my home network and how you can easily do the same. <br />
<br />
<b>Switching Internet Providers seamlessly...</b><br />
<b><br /></b>
Let me start by saying, yes after a few years of having Verizon FiOS, I went to the 'dark side' and became a Comcast internet customer. The main reason for my move was that 1) FiOS started charging double what I paid initially and 2) they were QoS'ing my traffic to Netfix / AWS. Instead of dealing with their limitations, I simply switched to Comcast. <br />
<br />
For most people, changing providers is a massive pain since now you have to re-ip or make some changes to some or all of your devices (new WiFi password, new IP's, firewall configurations, etc.) I learned my lesson from this switch, I will never ever use the cable company provided gateway device (modem + router), instead I will just get a standalone modem that I can plug into my firewall / router. Now, I control the firewall rules and all my network settings (DHCP, etc.) on hardware that I never have to rent or return. So, when Comcast jacks up my prices, I simply get a new modem for the next provider and seamless switch. No angry wife asking "What's the new WiFi password?!?"<br />
<br />
That's when I figured out that my little network setup made it really easy to move from FiOS to Comcast without having to make any changes to my network. Now, I can move from provider to provider without having re-architect my network.<br />
<br />
<b>Step 1 - ditch the cable company provided Gateway</b><br />
<b><br /></b>
After getting rid of the FiOS gateway, I was able to setup my pfSense firewall and simple wireless AP. All of the firewall intelligence is fully configurable and I even have an IDS/IPS (snort) as a part of the pfsense appliance. This little box has a ton of very cool and interesting tools, like Captive Portal (using simple username passwords to log into WiFi, making sharing it easier with guests.) You can even get NetFlow data using softflowd and an OpenVPN! (I will share how to set these up, but for now, please use the online documentation and google to set this up.)<br />
<br />
Let's start with the hardware:<br />
<br />
I bought this hardware new from Amazon in 2014 and it's been solid. You can get away with using older hardware, but for simplicity sake, here's what I used:<br />
<br />
Motherboard:<br />
<h1 class="a-size-large a-spacing-none" id="title" style="background-color: white; box-sizing: border-box; color: #333333; font-family: Arial, sans-serif; font-size: 21px !important; font-weight: normal; line-height: 1.3 !important; margin-bottom: 0px !important; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding: 0px; text-rendering: optimizelegibility;">
<span class="a-size-large" id="productTitle" style="box-sizing: border-box; line-height: 1.3 !important; text-rendering: optimizelegibility;"><a href="http://www.amazon.com/dp/B006ICQ3FK/ref=cm_sw_r_tw_dp_K.fWtb0P9JFN8T3V" target="_blank">BLKD2500CCE Intel Desktop Board D2500CC</a> -$120</span></h1>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6gMKJaateXk2ybxVcVMjgN4QtNgxA_tlMzErXXdriJJOhrR6ZAm08w4K4jH4JlwjZBzti2EXzqPBs-F5CVY5mpFs6qQx5xylwvhcqRbZ8GhQL6N_KcuO5paKWd8P2vIPuL0CTUNETykSL/s1600/motherboard.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6gMKJaateXk2ybxVcVMjgN4QtNgxA_tlMzErXXdriJJOhrR6ZAm08w4K4jH4JlwjZBzti2EXzqPBs-F5CVY5mpFs6qQx5xylwvhcqRbZ8GhQL6N_KcuO5paKWd8P2vIPuL0CTUNETykSL/s1600/motherboard.jpg" height="213" width="320" /></a></div>
<div>
Case:</div>
<div>
<h1 class="a-size-large a-spacing-none" id="title" style="background-color: white; box-sizing: border-box; color: #333333; font-family: Arial, sans-serif; font-size: 21px !important; font-weight: normal; line-height: 1.3 !important; margin-bottom: 0px !important; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding: 0px; text-rendering: optimizelegibility;">
<span class="a-size-large" id="productTitle" style="box-sizing: border-box; line-height: 1.3 !important; text-rendering: optimizelegibility;"><a href="http://www.amazon.com/gp/product/B005TX3LA4/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1" target="_blank">M350 Universal Mini-ITX enclosure</a> - $35</span></h1>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiStW3YqGo7PIaZpYjLkL81d_PrzJqBxA0EHwr-vKFtSPDZnBJww6vIdpC4oLp2tnmBiGCTtiOuXxmai7pQDPHqBTWWKO-naP8jUqnyCEzSvroi6eRP1ad5VSSp19Zhj8BOCghjAabC9eAu/s1600/case.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiStW3YqGo7PIaZpYjLkL81d_PrzJqBxA0EHwr-vKFtSPDZnBJww6vIdpC4oLp2tnmBiGCTtiOuXxmai7pQDPHqBTWWKO-naP8jUqnyCEzSvroi6eRP1ad5VSSp19Zhj8BOCghjAabC9eAu/s1600/case.jpg" height="197" width="320" /></a></div>
<div>
<span class="a-size-large" style="box-sizing: border-box; line-height: 1.3 !important; text-rendering: optimizelegibility;">(You'll need this <a href="http://www.amazon.com/gp/product/B000VE7GQQ/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1" target="_blank">power supply</a> and <a href="http://www.amazon.com/gp/product/B005TWE5E6/ref=oh_aui_detailpage_o02_s00?ie=UTF8&psc=1" target="_blank">plug</a>) $20</span></div>
<div>
<span class="a-size-large" style="box-sizing: border-box; line-height: 1.3 !important; text-rendering: optimizelegibility;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXlDAIv6Kr-T_zh3C0zEYJ8F0_3VFaA4ro9ZrtLrLwbs3H8rquefJasNHFeGahYTCLW-qmgF80necJNTBCnqlEG7YkYOqRtmdLwP3YOYNTk_AzW7znHvNJThEH0yzaduk8uBAsOaO3fabE/s1600/plug.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXlDAIv6Kr-T_zh3C0zEYJ8F0_3VFaA4ro9ZrtLrLwbs3H8rquefJasNHFeGahYTCLW-qmgF80necJNTBCnqlEG7YkYOqRtmdLwP3YOYNTk_AzW7znHvNJThEH0yzaduk8uBAsOaO3fabE/s1600/plug.jpg" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4ixQa6K5wEfp4MEE84ByKfJCReASbnRgEhyB8kZeVI_jpEFERkoKsm6FGeVhCKAEqO0tSHL_3bjKiWa51AP20K7wmP7Ncl6fTOeQItoBCX-4-vdvA1Vx2uvFQjknfLhplOvG76Z9Quakv/s1600/plug2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4ixQa6K5wEfp4MEE84ByKfJCReASbnRgEhyB8kZeVI_jpEFERkoKsm6FGeVhCKAEqO0tSHL_3bjKiWa51AP20K7wmP7Ncl6fTOeQItoBCX-4-vdvA1Vx2uvFQjknfLhplOvG76Z9Quakv/s1600/plug2.jpg" /></a></div>
<div>
<span class="a-size-large" style="box-sizing: border-box; line-height: 1.3 !important; text-rendering: optimizelegibility;"><br /></span></div>
<div>
<span class="a-size-large" style="box-sizing: border-box; line-height: 1.3 !important; text-rendering: optimizelegibility;"><br /></span></div>
4GB of RAM - PC3-10600 204-PIN SODIMM - <a href="http://www.amazon.com/gp/product/B004H6Y7DO/ref=oh_aui_detailpage_o02_s01?ie=UTF8&psc=1" target="_blank">Here's the one I used</a>. - $30<br />
4GB CF Card -<a href="http://www.amazon.com/SanDisk-SDCFH-004G-A11-ULTRA-Retail-Package/dp/B00065ANYW/ref=sr_1_4?s=electronics&ie=UTF8&qid=1405116714&sr=1-4&keywords=4gb+cf" target="_blank"> Here's the one I had laying around</a>. -$30<br />
<br />
Lastly, you'll need this <a href="http://www.amazon.com/Syba-Connectivity-Compact-Flash-Adapter/dp/B0036DDXUW/ref=sr_1_1?ie=UTF8&qid=1405117718&sr=8-1&keywords=cf+to+sata" target="_blank">CF-to-SATA adapter</a> - $13<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrwRBqAPV322gQ3XZv31N1FWQxUSvJIzgNfOlN59Y6lp-Q9y9g7flUm35IrlSamORe26m3sH0dOkxSgtndhqn755jlPCZ0QinqPJJgWSwwrP-oQrt3SJEwKOHIjC09lKiAoMe-EBJdFEvw/s1600/cfsata.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrwRBqAPV322gQ3XZv31N1FWQxUSvJIzgNfOlN59Y6lp-Q9y9g7flUm35IrlSamORe26m3sH0dOkxSgtndhqn755jlPCZ0QinqPJJgWSwwrP-oQrt3SJEwKOHIjC09lKiAoMe-EBJdFEvw/s1600/cfsata.jpg" height="273" width="320" /></a></div>
<br />
<br />
(Yes, you can use the 1GB card, but then you'll possibly limit your ability to install the packages you want.)<br />
<br />
<b>Total cost ~$250</b><br />
<br />
<b>Now let's build the pfsense firewall</b><br />
<br />
I built mine using the cfcard install on my Mac OSX. Here's how you do it.<br />
<br />
First, download the correct version for your hardware - 64bit 4GB Embedded CF Card.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL80Y3iGPa_08GpKj0UPaneiHFxiUWlNSV4B1C4QtL9BguxBOL6vCjUeScluJkUbD8EESYwvbGiWEjgX4w5gPF-sfbwcLwErS4G7midjODsn7KtBpoXYIwiYjkNqK7vMiz9NYYnI9bcvP1/s1600/Screen+Shot+2014-07-11+at+6.24.50+PM+1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL80Y3iGPa_08GpKj0UPaneiHFxiUWlNSV4B1C4QtL9BguxBOL6vCjUeScluJkUbD8EESYwvbGiWEjgX4w5gPF-sfbwcLwErS4G7midjODsn7KtBpoXYIwiYjkNqK7vMiz9NYYnI9bcvP1/s1600/Screen+Shot+2014-07-11+at+6.24.50+PM+1.png" height="320" width="296" /></a></div>
<br />
<br />
Now, let's get the image on the CFCard. You can either follow the documentation on the <a href="https://doc.pfsense.org/index.php/HOWTO_Install_pfSense" target="_blank">pfsense website</a>, or you can just run the following command:<br />
<br />
First, let's get the right disk, that is your CF Card:<br />
host:~> diskUtil list<br />
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
/dev/disk0</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
#: TYPE NAME SIZE IDENTIFIER</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
0: GUID_partition_scheme *251.0 GB disk0</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
1: EFI EFI 209.7 MB disk0s1</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
2: Apple_CoreStorage 250.1 GB disk0s2</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
3: Apple_Boot Recovery HD 650.0 MB disk0s3</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
/dev/disk1</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
#: TYPE NAME SIZE IDENTIFIER</div>
<br />
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
0: Apple_HFS MacHD *249.8 GB disk1</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
/dev/disk2</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
#: TYPE NAME SIZE IDENTIFIER</div>
<div style="background-color: black; color: whitesmoke; font-family: Monaco; font-size: 10px;">
0: Apple_HFS USB *4 GB disk2</div>
<br />
<br />
dd if=/path/to/pfsense.img of=/dev/disk2<br />
<br />
Here's the <a href="http://www.thelinuxdaily.com/2010/01/writing-images-to-disk-on-mac-osx-with-dd/" target="_blank">source</a> I used to make my disk. <br />
<br />
<b>Awesome, my pfsense firewall is on my CF Card, now what?</b><br />
<b><br /></b>
Let's install the hardware and get the firewall online. Follow the prompts, and the online documentation from pfsense to complete the install. For this example, we are just going to install the WAN and LAN links, if you want to build your own VLAN's, you can read the fine manual to do that. <br />
<br />
<b>I want the syslogs!</b><br />
Instead of logging the data directly to my pfsense firewall, I decided to use a Raspberry Pi. You do not need to do this step, and you can feed your syslog directly into your Splunk Indexer. I did this to setup a forwarder and also because I had an extra Raspberry Pi.<br />
<br />
Raspberrypi - syslog-ng<br />
<br />
<b>To the Cloud!</b><br />
<br />
Now let's setup an EC2 instance on Amazon and use the t1-micro instances to setup our Splunk environment in the Cloud. From here, you should be able to simply install Splunk on the Amazon AMI's and start Splunking your data! I plan on doing a post on how to setup your environment and also help you setup a mobile site using the <a href="http://www.splunk.com/download/mobile_access_server" target="_blank">new Splunk mobile server. </a>amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com0tag:blogger.com,1999:blog-4935085660130283180.post-59292368822740703862014-02-08T17:24:00.005-08:002014-02-08T17:24:59.458-08:00Home Monitor v.3.0 - Working on it and having it done shortly!To all the people who have downloaded Home Monitor, THANK YOU! I am currently working on a much overdue upgrade (v.3.0) to Home Monitor App for Splunk. I should be having it out soon, needless to say getting married and taking time off puts a bit of a dent in my 'free time.' Thank you all for the posts, emails etc., I made sure to pay attention to all of them and will be incorporating many of the changes to the v.3.0. <br />
<br />
A couple things, the 3.0 version will be a Splunk 6.0+ only version, so if you have not already, upgrade your Splunk instance to v.6.0+ . Also, I'm going to work on a bandwidth calculator, given all the press that Verizon has had around the NetFlix bandwidth throttling. This way people will have an easy way to show that their bandwidth that they are paying for is being throttled. It will be a Linux (Mac OSX) script first, and I will work on a Windows version as well, but that will happen in a future release. <br />
<br />
Other than that, the Google Maps will be replaced with the native Splunk Maps visualization and I will have wider support for other routers, including pfsense firewalls. Honestly, I've stopped using the Verzion FiOS router and put a pfsense firewall in place instead. In my opinion, it's a better solution and easier to customize (think VPN, proxy, IPS/IDS, etc.) and you can actually close all the ports on your firewall that you want without having to keep the admin port open. (See <a href="http://forums.verizon.com/t5/FiOS-Internet/Guy-accessed-remote-administration-port-4567-on-my-router-Thanks/td-p/241017">http://forums.verizon.com/t5/FiOS-Internet/Guy-accessed-remote-administration-port-4567-on-my-router-Thanks/td-p/241017</a>) <br />
<br />
Thanks again for all the posts and support, I'll be having the latest version out shortly and feel free to leave comments or suggestions on other features you want to see in later versions. <br />
<br />
Thanks,<br />Kamamiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com2tag:blogger.com,1999:blog-4935085660130283180.post-47913387454098413352013-08-08T10:23:00.002-07:002013-10-30T11:12:41.077-07:00Update: Support for Netgear, Asus and Skyhub, need beta testers!Hello everyone, I was able to quickly build some support for the routers from the sample data which was sent to me. Here are the default (fios) sourcetype, but I've now included the Asus, NetGear and Skyhub routers. Please feel free to add this props.conf file in your :<br />
<b>$SPLUNK_HOME/etc/apps/homemonitor/local/props.conf </b><br />
<br />
Once you've added the entries, then just restart your Splunk instance.<br />
<br />
If you have already added the UDP source, then simply remove it and re-add it and do the following steps.<br />
<br />
Set the UDP port: 514<br />
Set sourcetype: From list<br />
Select source type from list *<br />
<select your router><br />
fios, asus, netgear, or skyhub<br />
<br />
<br />
**NOTE** Some dashboards will not work like the FiOS router, but a majority of them should still work.<br />
<br />
Once I have some validation that these extracts work, then I will include them in the v3.0 Home Monitor release.<br />
<br />
------------------- props.conf -----------------<br />
<br />
[fios]<br />
# These extracts are intended for use with the MI424WR-GEN2 Verizon FiOS Home Router / Firewall. The firmware version that has been tested is 20.19.8.<br />
EXTRACT-action = (?i) .*?: (?P<action>\w+)(?= )<br />
EXTRACT-transport = (?i) .*? \( : (?P<transport>\w+)(?= )<br />
EXTRACT-src_ip = (?i)\(.*? (?P<src_ip>\d+\.\d+\.\d+\.\d+)(?=:)<br />
EXTRACT-src_port = (?i)^(?:[^:]*:){7}(?P<src_port>[^\]]+)<br />
EXTRACT-dest_ip = (?i)^[^>]*>(?P<dest_ip>[^:]+)<br />
EXTRACT-dest_port = (?i)^(?:[^:]*:){8}(?P<dest_port>[^ ]+)<br />
EXTRACT-remote_ip = (?i)^(?:[^:]*:){6}\d+\s+\[(?P<remote_ip>[^:]+)<br />
EXTRACT-remote_port = (?i)^(?:[^\[]*\[){2}\d+\.\d+\.\d+\.\d+:(?P<remote_port>[^\]]+)<br />
EXTRACT-connection_state = (?i)^[^\]]*\]\s+(?P<connection_state>\w+\s+\w+)<br />
EXTRACT-config_change_user = (?i) user (?P<config_change_user>[^ ]+)<br />
EXTRACT-process = (?i) .*? (?P<process>[a-z]+)(?=:)<br />
pulldown_type = 1<br />
<br />
[asus]<br />
# Based on Asus RT-N66U router syslog output.<br />
FIELDALIAS-dst = DST as dest_ip<br />
FIELDALIAS-dpt = DPT as dest_port<br />
FIELDALIAS-proto = PROTO as transport<br />
FIELDALIAS-SPT = SPT as src_port<br />
FIELDALIAS-SRC = SRC as src_ip<br />
EXTRACT-action = (?i) .*?: (?P<action>\w+)(?= )<br />
pulldown_type = 1<br />
<br />
[netgear]<br />
# Based on Netgear FV318N router syslog output.<br />
FIELDALIAS-dst = DST as dest_ip<br />
FIELDALIAS-dpt = DPT as dest_port<br />
FIELDALIAS-proto = PROTO as transport<br />
FIELDALIAS-SPT = SPT as src_port<br />
FIELDALIAS-SRC = SRC as src_ip<br />
EXTRACT-action = (?i) LOG_PACKET\[(?P<action>[^\]]+)<br />
pulldown_type = 1<br />
<br />
[skyhub]<br />
# Based on Skyhub SR101 router syslog output.<br />
FIELDALIAS-dst = DST as dest_ip<br />
FIELDALIAS-dpt = DPT as dest_port<br />
FIELDALIAS-proto = PROTO as transport<br />
FIELDALIAS-SPT = SPT as src_port<br />
FIELDALIAS-SRC = SRC as src_ip<br />
EXTRACT-action = (?i) kernel: (?P<action>[^\-]+)<br />
pulldown_type = 1amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com13tag:blogger.com,1999:blog-4935085660130283180.post-34748837516693964492013-04-27T05:55:00.001-07:002013-04-27T05:55:45.541-07:00Requests for routers or firewall logs...I just wanted to update everyone that I am working on a Technology Add on for the home monitor app that will cover most of the other popular routers and firewalls. As of now I have a pfsense Sourcetype, and will be working on various other source types. That's where I need your help. Can you send me a sample of your log file so I can setup the field extractions properly? I'm also going to see about putting a poll up on this site to see what are the most popular routers out there.<br />
<br />
Stay tuned for more updates on this app and thanks for the support so far!amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com2tag:blogger.com,1999:blog-4935085660130283180.post-46845999121732678692012-09-10T09:21:00.001-07:002013-04-29T13:13:29.797-07:00My Splunk Applications PlaygroundHere are some applications I'm working on to be used with Splunk (free download at http://www.splunk.com).<br />
<br />
Thanks,<br />
Kam > amiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com2tag:blogger.com,1999:blog-4935085660130283180.post-15185050013179624842012-09-10T09:19:00.000-07:002013-04-29T13:13:19.649-07:00Home Monitor for Splunk v.2.2I've just released my first Splunk Application called Home Monitor. It basically runs on core Splunk and gives you some nice dashbaords, reports and views about the traffic going through your home network. The application relies on syslog data that is being tracked by your home router. This data is then sent into your Splunk instance and you can then see what's happening in your network.<br />
<br />
Here are the <a href="http://amiracle19.blogspot.com/p/setting-up-verizon-fios-router-for-home.html">step by step</a> instructions on how to setup your Verizon FiOS router and Splunk. Here's the link http://amiracle19.blogspot.com/p/setting-up-verizon-fios-router-for-home.html .<br />
<br />
This latest release has some new pages and dashboards designed to allow you better insight into your home router configurations. The page "Router Configuration" lets you know if any changes have occurred in the last 30 and 7 days along with a real time check to alert you to change currently happening. <br />
<br />
The duration and traffic trends pages give you an insight into what kind of traffic is being brought into your network and which clients are being too chatty. The Duration page gives you break down by exploitable ports which you might want to guard against. Lastly, the same duration page lets you know if any of your machines have been compromised since most desktops should not be connecting for longer than a few seconds or minutes to outside sources. <br />
<br />
<br />
I have not tested this on the new Verizon FiOS "N" Router but I plan on trying to get access to one soon.<br />
<br />
Here is the link to the <a href="http://youtu.be/pgJ4dtIn5wo" target="_blank">youtube video</a> showing you how to install and use the app : http://youtu.be/pgJ4dtIn5wo . Let me know if you have any questions about the video.<br />
<br />
<br />
Enjoy,<br />
Kamamiraclehttp://www.blogger.com/profile/05644628158194263819noreply@blogger.com25