Mining Packets Via The Command Line
If you're considering becoming an intrusion analyst, there are two other sources of data that go hand in hand with your IDS/IPS: Packet data and logs. A NIDS, or network intrusion detection system, inspects packets as they cross the network whereas a HIDS, host-based intrusion detection system, resides on a server and inspects logs, files and so forth. A NIDS will capture the packets that triggered it's rule or signature and can be configured to capture additional packets from the session, but it will not capture all of the packets flowing across the network, nor should you want it to. The overhead of capturing all of the packets for reconstruction and investigation is best left to another device dedicated to that purpose. There are many types of programs that can help you reconstruct packets of interest. One of the best is Moloch, a full blown project using Elasticsearch as the back end indexer, with a mature Web based interface that has a rich set of built in queries. Whatever you use, the same principles apply.
Assuming you're using a tool like tcpdump, snoop or daemonlogger, you have a directory somewhere on your packet auditor with a lot of pcap-compatible capture files. Your goal is to pull out the packets you need from this (probably) huge archive and inspect them or run them through your tool of choice.
The first step is to identify the capture file or files that have the packets you need to look at. Once you do that you can extract them into a separate packet capture that you can run through your pcap-compatible tools or upload to another box as needed.
Start building your filter by running the ls command with the -l parameter to see the long listing.
[jeffsoh@packet1 capture_dir]$ ls -l
-rw-r--r--. 1 root root 1.1G Jun 7 01:25 mycaptures.pcap.1370572245
-rw-r--r--. 1 root root 1.1G Jun 7 06:03 mycaptures.pcap.1370582729
-rw-r--r--. 1 root root 1.1G Jun 7 08:28 mycaptures.pcap.1370599389
-rw-r--r--. 1 root root 1.1G Jun 7 09:20 mycaptures.pcap.1370608087
...
Now grep for the date. Let's assume our packets of interest occured on June 8th.
[jeffsoh@packet1 capture_dir] $ ls -l | grep 'Jun 8'
-rw-r--r--. 1 root root 1073743705 Jun 8 03:43 mycaptures.pcap.1370660768
-rw-r--r--. 1 root root 1073744701 Jun 8 07:08 mycaptures.pcap.1370677399
-rw-r--r--. 1 root root 1073742386 Jun 8 08:02 mycaptures.pcap.1370689727
-rw-r--r--. 1 root root 1073743170 Jun 8 08:30 mycaptures.pcap.1370692931
-rw-r--r--. 1 root root 1073741832 Jun 8 09:40 mycaptures.pcap.1370694616
-rw-r--r--. 1 root root 1073741889 Jun 8 10:35 mycaptures.pcap.1370698825
-rw-r--r--. 1 root root 1073744142 Jun 8 11:21 mycaptures.pcap.1370702122
-rw-r--r--. 1 root root 1073742208 Jun 8 13:55 mycaptures.pcap.1370704877
-rw-r--r--. 1 root root 1073741860 Jun 8 22:23 mycaptures.pcap.1370714127
[jeffsoh@packet1 capture_dir]$
And the alert occurred at 8:05 AM. We see a capture with a timestamp of 8:30; that's probably what we need. We run stat on the file to check and make sure. We could also use the excellent Wireshark tool capinfos.
[jeffsoh@packet1 capture_dir] $ stat mycaptures.pcap.1370692931
File: `mycaptures.pcap.1370692931'
Size: 1073743170 Blocks: 2097168 IO Block: 4096 regular file
Device: 804h/2052d Inode: 120588319 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-06-08 08:02:11.527770131 -0400
Modify: 2013-06-08 08:30:16.324924184 -0400
Change: 2013-06-08 08:30:16.324924184 -0400
We have packets from between 8:02 and 8:30. We could just use that packet capture, but if we are looking at an alert from an IDS, we have the timestamp of when the system saw the packet. Depending on the length of the session, this capture may not give us the complete session, so we can pull both packets from that hour. Were this a busier time of day or network we might have dozens of packets for that hour.
[jeffsoh@packet1 capture_dir] $ ls -l | grep 'Jun 8 08:'
-rw-r--r--. 1 root root 1073742386 Jun 8 08:02 mycaptures.pcap.1370689727
-rw-r--r--. 1 root root 1073743170 Jun 8 08:30 captures.pcap.1370692931
Now that we've built our grep string for the time frame we need, all we have to do is use it in a for loop to set the variable for tcpdump to loop through.
Up arrow to pull up our last command and add the for loop around it.
for i in $( ls -lah | grep 'Jun 8' | grep '10:' | awk '{print $9}' );do tcpdump -nn -r $i 'host 10.80.91.43' -w /root/workspace/$i.extract.pcap;done
We now have a pcap for investigation with all of the packets captured for that IP address.
