The Power of tshark, Part 3

All tshark/Wireshark dislay filters aren't limited to a precise match. tshark has the powerful ability to find multiple values that match one part of a field. A good example is the http.request.uri field. There is a modifier called "contains" that allows you to search the field for matches in diverse content. In the http.request.uri field, we could use "http.request.uri contains "google"", allowing us to match on tools.google.com, apis.google.com, safebrowsing, www, mail, news and so forth and domains that contain the word google even if it's part of a longer word, like googlesyndication.com.

tshark -n -r packets1.pcap -Y "http.request.uri contains "google""

132   0.026293 10.10.10.2 -> 10.10.220.100 HTTP 159 CONNECT talkx.l.google.com:5222 HTTP/1.1
247   0.048647 10.10.10.2 -> 10.10.220.100 HTTP 243 CONNECT talkx.l.google.com:5222 HTTP/1.1 , NTLMSSP_NEGOTIATE
23956   8.625725 10.20.122.52 -> 10.10.220.100 HTTP 298 CONNECT pagead2.googlesyndication.com:443 HTTP/1.1
28461  11.024309 10.20.121.95 -> 10.10.220.100 HTTP 273 CONNECT safebrowsing.google.com:443 HTTP/1.1
29078  11.308659  10.20.42.17 -> 10.10.220.100 HTTP 396 CONNECT clients1.google.com:443 HTTP/1.0
38350  19.076371 10.20.112.137 -> 10.10.220.100 HTTP 123 CONNECT tools.google.com:443 HTTP/1.1
38368  19.096608 10.20.112.137 -> 10.10.220.100 HTTP 207 CONNECT tools.google.com:443 HTTP/1.1 , NTLMSSP_NEGOTIATE
51468  25.387301 10.30.130.104 -> 10.10.220.100 HTTP 520 GET http://www.googletagservices.com/tag/js/gpt.js HTTP/1.1

The Power of tshark, Part 2

To expand on our example looking for ICMP type 3 packets, let's narrow that filter down to one specific type of Destination Unreachable message. If we look at the IANA ICMP parameters list, found at here, we see that there 15 codes that can be set with type 3. Some of the more common ones are code 4, Fragmentation Needed and Don't Fragment was Set, code 7, Destination Host Unknown and code 10, Communication with Destination Host is Administratively Prohibited. Let's add a code 4 to our tshark diplay filter.

tshark -r packets1.pcap -Y "icmp.type == 3 and icmp.code == 4"
501732 194.413516  10.10.10.1 -> 10.10.20.1 ICMP 70 Destination unreachable (Fragmentation needed)
507176 196.247873  10.10.10.1 -> 10.10.20.1 ICMP 70 Destination unreachable (Fragmentation needed)

Our output looks the same, with one difference. We're now seeing only ICMP messages that are type 3 and code 4, instead of all destination unreachables.

We've been using the default fields that tshark displays. But we can specify which fields to see, if we wish. In the case of ICMP there wasn't much reason to, as it is a concise output and shows us just what we need anyway. But when looking at other types of packets, we might want to limit the fields to specific data we need, or we may be looping through a large number of packets and pulling out just certain fields we wish to report on, like the IP addresses that generate a certain HTTP status code or just the IP's that generated traffic to a certain port or host.

tshark -n -r packets1.pcap -Y "tcp.port == 80"

16714   6.578480 192.203.136.227 -> 10.10.20.2 HTTP 471 HTTP/1.1 200 OK  (text/html)
16715   6.579366 10.10.10.1 -> 10.10.20.1 TCP 60 3786?80 [ACK] Seq=221 Ack=1846 Win=65071 Len=0
16716   6.579611 10.10.10.1 -> 10.10.20.1 TCP 60 3786?80 [FIN, ACK] Seq=221 Ack=1846 Win=65071 Len=0
16717   6.580180 10.10.10.1 -> 10.10.20.1 TCP 62 3787?80 [SYN] Seq=0 Win=65535 Len=0 MSS=1380 SACK_PERM=1
16720   6.582334 140.172.17.191 -> 10.10.20.2 TCP 1434 [TCP segment of a reassembled PDU]
16721   6.582384 140.172.17.191 -> 10.10.20.2 TCP 1434 [TCP segment of a reassembled PDU]
16722   6.582828 140.172.17.191 -> 10.10.20.2 TCP 1434 [TCP segment of a reassembled PDU]
16724   6.583013 140.172.17.191 -> 10.10.20.2 TCP 1434 [TCP segment of a reassembled PDU]
16725   6.583020 140.172.17.191 -> 10.10.20.2 TCP 1434 [TCP segment of a reassembled PDU]
16729   6.583194 10.10.20.2 -> 140.172.17.191 TCP 60 [TCP Window Update] 45932?80 [ACK] Seq=882 Ack=113388 Win=17817 Len=0

To filter this down to only see source IP and port and destination IP and port, we need to tell tshark we want to display only certain fields, using the "-T" parameter. We can specify fields, as well as output types of pdml, ps, psml or text.
After the -T fields param, we'll use the "-e" parameter to specify which fields to display. The Source IP field is "ip.src", the Source Port field is "tcp.srcport" and the destination IP and port are, as expected, "ip.dst" and "tcp.dstport".

So after adding these filters, our output is narrowed down to the four fields of interest for this run.

tshark -n -r packets1.pcap -T fields -e ip.src -e tcp.srcport -e ip.dst -e tcp.dstport -Y "tcp.port == 80"
10.10.10.1   19248   66.175.58.9     80
10.10.10.1   19248   66.175.58.9     80
10.10.20.1   80      10.85.120.101   3743
192.203.136.227 80      10.10.10.2   45870
74.125.69.95    80      10.10.10.1   36719
74.125.69.95    80      10.10.10.1   36719
74.125.69.95    80      10.10.10.1   36719
10.10.10.1   5925    184.169.162.197 80
10.10.10.1   5925    184.169.162.197 80
10.10.10.2   45870   192.203.136.227 80

Next post we'll use more display filters to build our custom output and look at tsharks ability to do stats from the command line.

The Power of tshark

tshark (the command line version of Wireshark), is a powerful tool for doing network forensics and investigation. For the new intrusion analyst, the GUI may be more comfortable to work in, but as powerful as Wireshark is, it has it's limitations. The larger the pcap file, the longer it will take to load into Wireshark and it's possible to crash the program or even your operating system with too large of a file.

Another limitation is getting those pcaps transferred to you analysis machine, especially if the time frame you need to investigate spans multiple files (or hours or days or weeks).

tshark is uniquely suited to parsing through large amounts of packet data, across multiple files, with all the protocol analyzing power of Wireshark AND allowing you to use every one of Wiresharks 174,000+ display fields.

If you've never worked with tshark before, don't worry. You can start out with some very basic tcpdump-like syntax and then add more filtering and/or fields to display as you go along. 

Before you start, I'd suggest bookmarking the Display Filter Reference for Wireshark, found at here.

There's an even easier way to discover what filters you want to use by actually using a running instance of Wireshark, but I'll address that later. Let's keep it in the dark spaces for now.

A simple tshark command to show you all packets with an ICMP type of 3 (destination unreachable) would be like this:

tshark -r packets1.pcap -Y "icmp.type == 3"

Simple enough. The -r means read a file in instead of sniffing an interface (the same as tcpdump), packets1.pcap is our already captured packet data file, and -Y means use this display filter, which is "icmp.type == 3". 

And here's a sample of what tshark shows:

501732 194.413516  10.10.1.10 -> 10.10.2.10 ICMP 70 Destination unreachable (Fragmentation needed)

tshark also shows us the ICMP code as well for no extra cost. As we can see the destination unreachable packet was because fragmentation was needed but the Don't Fragment flag was set (refer to IANA's ICMP Parameters doc and look up ICMP Code 4 for Type 3 here.

Had we wanted to see echo requests instead destination unreachables, we would change the "3" to an 8. Echo replies would be a "0" and so forth.

Now we are seeing all default fields coming back from tshark and using just one filter. But we can specify what fields we'd like to see instead. By using the -T parameter, we can specify what we want tshark to show us: the fields, specified with the -e parameter used in conjunction with -T, pdml output (Packet Details Markup Language), ps for Postscript, psml for Packet Summary Markup Language or text, which is the default if nothing is specified. In the next command we're going to tell tshark we'd like to see ONLY the following fields:

The source IP
The destination IP
The ICMP type
The ICMP code
and only for ICMP packets of type 3, destination unreachable.

tshark -r packets1.pcap -T fields -e ip.src -e ip.dst -e icmp.type -e icmp.code  -Y "icmp.type == 3"

And our output looks like this:

10.83.40.10,10.83.220.100            10.83.220.100,10.84.21.32           3       4
10.83.40.10,10.83.220.100            10.83.220.100,10.84.21.32           3       4
10.83.204.80,205.251.192.1          205.251.192.1,10.83.204.80         3       3
10.83.220.103,10.82.12.95            10.82.12.95,10.83.220.103           3       3
10.83.204.80,124.232.142.220      124.232.142.220,10.83.204.80     3       3

Now we could create a dataset with all the destination unreachables along with the code, telling us what type of unreachable messages they are from large pcaps of data. A 'for' loop would allow us to iterate through as many pcaps as needed.

Next post, we'll use more of the display filters available, add some labeling and get our data ready for importing into a spreadsheet or doing some graphing.

nmap Run Time Interaction

Even if you've only been in network security a short time, you've probably come across nmap by now, the port scanner written by Fyodor (Gordon Lyon). It's been around a long time and is the de facto standard for reliable and feature rich port scanning.

You may have installed it and played with it or maybe you're now using it in your job duties. nmap is up to version 6.40 and has a whole slew of options including scripting that can do so much more than just return what ports are open on a box.

But once you've carefully consulted the man page and crafted that command to do just what you need and hit Enter, did you know you can still interact with your scan?

nmap has several run time interaction functions that can give you additional insight into your scan. For example, if you omitted verbosity flags (-v or -vv or even -vvv) to show you more of what the program is doing, you can increase the level while the scan is running by hitting the "v" key. Hitting it a second or third time increases the verbosity each time. Too much data? Hitting the capital "v" key will decrease verbosity each time you use it, all the way back down to the original run time level.

If you're having issues with a scan and not getting the right results, you can use the "d" key to turn up the debug level. Like verbosity, each subsequent use of the key increases the level and and using the upper case version moves it back down.

"p" will turn on packet tracing, showing you a tcpdump-like log of each packet. For this one, there's (obviously) only one level and capital "p" will turn it back off.  Your packet trace will look similar to this:

SENT (14.1570s) TCP 10.82.250.105:59073 > 10.82.250.7:777 S ttl=42 id=54028 iplen=44  seq=2857954232 win=3072
SENT (14.1570s) TCP 10.82.250.105:59072 > 10.82.250.8:451 S ttl=52 id=48876 iplen=44  seq=2858019769 win=1024
RCVD (14.1570s) TCP 10.82.250.8:451 > 10.82.250.105:59072 RA ttl=128 id=14308 iplen=40  seq=0 win=0
SENT (14.1600s) TCP 10.82.250.105:59073 > 10.82.250.7:578 S ttl=53 id=64554 iplen=44  seq=2857954232 win=2048
SENT (14.1600s) TCP 10.82.250.105:59072 > 10.82.250.8:112 S ttl=41 id=33103 iplen=44  seq=2858019769 win=2048
SENT (14.1600s) TCP 10.82.250.105:59072 > 10.82.250.11:729 S ttl=37 id=37013 iplen=44  seq=2858019769 win=2048
RCVD (14.1600s) TCP 10.82.250.8:112 > 10.82.250.105:59072 RA ttl=128 id=14309 iplen=40  seq=0 win=0
SENT (14.1620s) TCP 10.82.250.105:59072 > 10.82.250.11:54 S ttl=49 id=30436 iplen=44  seq=2858019769 win=2048