More NetWitness Investigator

To import packets into Investigator, you need to create a collection (the default collection, as we saw, is simply called "Demo Collection". Create a new collection (Ctrl-L or use the menu), and then you can import your pcap file. The capture must be 1 GB or less, and you can have 25 simultaneous captures at any one time.
Double click your new collection, and after a second or so, you should see it's status change to ready. Now you can right-click on it and choose "Import Packets". Browse to where you've saved your pcap file (hopefully you captured it with full packet data), and you'll see the import process begin. Depending on how much of your 1 GB limit you used, this may take a little while. Once you get the "ready" status once more, double click your collection once more and open it up..

Here we see all of our source and destinations, aliases, content types, ports and services used... and in the lower part of the report, (below) we have counties and cities involved (mine always has Beijing, as does probably yours) and organizations.

Since we see some of our hourly packets from China there, we can now drill down and see that kind of traffic they entail. Clicking on the number in parentheses (the packet count) after Beijing takes us to a nice sessions screen with our summaries for each packet...

I'm redacting my IP address here, replacing it with x.x.x.x, but you get the idea.. Now I can zoom in on the packet data.. and see that as we mouse over each field in the packet header, we get a little pop up telling us the field and doing the hex conversion for us.

We have all the summary information of all packets in the capture to look at. We can, in a few mouse clicks, go from the high overhead view down to the single packet, to any field in that packet, and see it's identification and conversion, if needed. Pulling all this information out manually, then drilling down to granular of a level, even with scripting, would take far, far longer.
Next we need to explore using Investigator to do the data capture itself.

Another Free Packet Tool - NetWitness Investigator

NetWitness Investigator is a  free tool for packet analysis. This one runs on Windows (including Windows 7) only (except for the commercial version, which runs on Linux) and has functionality that includes wireless support, real time layer 7 analysis, IPv6 support, SSL decryption (with the server cert) and full context searching including regex. The freeware version license supports 25 simultaneous 1 Gb captures.

To get started, grab a copy from the download page here and install.
When opening the program for the first time, you'll be prompted to login, or create an account. Go ahead and create a community account and after activating Netwitness (from the email you'll receive), you'll be presented with the Investigator window. In the left hand pane will be a folder marked Demo Collection, a packet capture that allows you to test out the app immediately, without needing to do any capturing of your own.

Double-click the Demo collection icon, and a new tab will open with the contents of that collection, which will look like this:

We can see a whole host of attributes that can be drilled into to explore. At the very top, we see Investigator has flagged three alerts for us, non-standard http, cleartext passwords and irc file transfer. Clicking the "Alert" attribute will drill down to the next level where all the attributes are related to these three alerts. We see the services used, source and destination addresses, ports and protocols, and all our layer 7 info broken out for us, like email addresses, user accounts, and attachments.

Clicking on the "sample_vulnerability:cleartextpasswords:other (1)" link takes us down another level, showing us all the attributes of the packets that triggered this alert. We now know the IP addresses involved, the service type (POP3), the user account involved and the email address involved. When we drill down even further in our later 7 data, Investigator will do reconstructions for us. For example, the email address attribute has one session associated with it (the number in green in parentheses after the field shows the number of sessions). When we click on the number after the email address, NetWitness reconstructs the email header for us:

   
We can do the same for the other data presented, such as the Creditcards.txt attachment. Investigator will ask us how we want to open the file, allowing us to open it in a default app for the data type, choose a different app or use Investigator itself to open it. 


This is just the very basic starting point of what you can do with this tool. Like xtractr, it's a very good tool to investigate a capture and you need to drill down deep in a visual way to ascertain what action took place. 

BSides Cleveland

Went to Security BSides Cleveland yesterday (February 18) and had a great time. The conference was at the House of Blues in downtown Cleveland, and there was a great lineup of speakers. You could truly say there was something for everyone, because the topics ran the spectrum of everything from security education and GRC (high level talks) to thicknet (SQL injection), using Netflow for security and claims based identity management.
I'm definitely looking forward for the next BSides that comes within range of me attending.
The videos are up and available at Ustream (http://www.ustream.tv/channel/security-justice---test)

Learning awk with Practical Examples

Learning awk with Practical Examples Ten Days of Awk at bashshell.net. I just found this site today. It looks like it will be a good resource for beginners at Linux (including those just moving over to NetSec).

OpenFPC

OpenFPC Network traffic capture tool. Three components here. A client device used by the analyst to request a packet capture. A master device (or devices) that receive the request, look up what device would have seen the traffic and pull the capture file and display it back to the client. One or more slaves, which are the sniffers doing the packet captures. Looks very interesting; added to my growing list of tools I'd like to test..

BPF's For Beginners

If you've just started looking at packets, no doubt you've already found the need to pull out packets based on more than just the primitives built into tcpdump and other pcap aware tools (such as src, host, port, net, etc). tcpdump supports the use of the Berkeley Packet Filters expressions, allowing you to search on any field that tcpdump records. The basic syntax is field : operator : operand. For example, if you wished to see all IP packets whose length is greater than 20 (meaning IP options are in use) you would add the filter:
 'ip[0] & 0x0F > 5' This says to look at the first field in the IP header (offset 0, headers begin counting at 0), and using a bitmask, filter out the first nibble (the first four bits, 0), looking at the last four bits, (F) and show any that are greater than 5. We know the IP header length field is expressed as a 32 bit word (four bytes), so a 5 in that field (5 x 4 bytes) equals 20, the length of a normal IP header (also the minimum size). So any size greater than 5 would mean a header more than 20 bytes. This means IP options have been added to the end of the header, such as strict source routing, or loose source routing, rarely used and often a sign of a malicious attempt to route packets through a particular node for sniffing or interception.
   Enclosing your filter in quotes to avoid syntax errors is a good habit to get into. You'll eventually need them as your filters get more complex.
You can search on any field in any of the embedded protocols as well. If you needed to see SYN packets only, you could filter on the TCP header with a filter of 'tcp[13] = 0x02'. Here we would see only packets with a value of 2 in byte 13, which would have only the SYN flag set.
   If we needed to look at just a single bit or combination of bits, instead of a whole byte or a whole nibble as in the first filter, we would go back to our bitmasking. Bitmasking bits here is simply applying a bitwise AND  to filter out any bits we don't wish to look at, and preserving the bits we do. If we AND a zero to a bit, we end up with zero, if we AND a one, we end up with a 1. So if need to see all SYN-ACK packets, we would want to see all packets with byte 13 in the TCP header that have the least most significant bit set in in the first nibble and the second least significant bit set in the second nibble.

                    C E  U A   P R  S  F
                    1  1  1  1    1  1  1  1
Our mask     0  0  0  1    0  0  1  0

So our bitmask for SYN-ACK packets would be 0x12 and we would apply it as this:

'tcp[13] = 0x12' (no other flags should be set here as the SYN-ACK combination is only used in the TCP handshake.)

You can extend this to any field, down to a single bit, and chain them together to get as granular as you need. When you begin writing complex filters and using them over and over again, you can put each one in a file, and use the tcpdump -F parameter to load it, instead of retyping it each time. Very powerful stuff for dissecting packets, and very good to learn. You can't always be sure that you'll have access to Wireshark or some other protocol parser when you do to do network forensics. If you learn the command line way to do things first, you'll hopefully never get stuck.

If you want great training in bits and bytes, protocols and packets, and everything in between (especially if you want to be an intrusion analyst) I'd recommend looking at the SANS 503: Intrusion Detection In-Depth course. Taking it live, in my opinion, is always best, as you'll get the most interaction with the instructor, work with other folks interested in the same knowledge and absorb much more than just the class from the atmosphere of a SANS conference. But take it any way if you can, if you possibly can. And if you CAN go live, use the time you have there to its fullest. Sign up for the free SANS @Night classes, go to the Storm Center presentation, go listen to the keynote speakers in the evening, do the lunch and learns and sit in on a BOF (Bird of a Feather) session. That's why I'd never to go to a SANS conference at Disney World or Virginia Beach. Too many temptations to waste your time doing other things, and the opportunity to learn practically all day long is too valuable. But's that's just the way I roll. =-)

Packet Analysis With xtractr - Continued

Now that we have our packets indexed, we need to start the web service and load them. The default address to bind to is localhost, on port 8080. If you're working remotely on your xtractr box, and especially if you want others to be able to look at the packets with you, you'll want to change to bind to your networked interface (realize no one can connect to the box and look at the packets or run queries without a Mu Dynamics account).
So we now use the browse parameter, like thus: xtractr browse (index_directory) --host (network_ip_address).

So using our example above, and assuming our box has an IP address of 192.168.100.10, our browse command would be: xtractr browse index_dns --host 192.168.100.10
If we wanted to use a port other than 8080, we could add the --port parameter to bind to a port other than 8080.

Point your browser to ip_address_of_the_xtractr_box:8080 or, if you specified a different port, ip_address:(port_number). Your browser will be directed to the Mu Dynamics xtractr site, where you'll be prompted to login with that account you created at step 1. Now, as you query and drill down, your data will be streamed through the Mu site where all the processing will take place (again note that unless you choose to upload your pcaps, they will remain on your box.) 

Mu has a large repository of user supplied pcaps available to the community, to test with and experiment. They encourage you to contribute to the library, and there are some very obscure and esoteric packet captures available to you there.

You can go to the xtractr web site's live demo, at http://www.pcapr.net/xtractr/demo#/flows, and see what all this packet parsing goodness looks like, and play before you download.

(Folks at Mu.. if you come across this post and I've screwed up anything, please Twitter me @JeffSoh and I'll correct it ASAP)

Packet Analysis With xtractr

I've mentioned xtractr by Mu Dynamics before, and over the last week or so I've been using it more and more to look at EOI's. If you've not used it before, here's a little overview of this tool.

• xtractr runs on Linux (the latest version is 4.5.40426) and can be downloaded here. 
• You will need an account (free) with Mu Dynamics to use the query services.
• xtractr can be used in stand-alone mode, which means your pcaps, queries and labels never leave your machine. It can also be used Mu Studio to convert the data into a stateful test case.
• More than one person can look at the data at a time, and if you need to look at more than one capture at the same time, you can run multiple instances of xtractr.
• The free lite version can index a capture of either 10 million packets or 1 Gig of pcaps.

That said, here's what you'll need to do to use xtractr. After creating your account, and downloading the tarball and installing, you're ready to index your packets. You'll want full length packet captures here if you're going to do network forensics.

Create yourself a workspace directory of whatever name you want, and copy (or capture, if you're testing) your pcap there. I'd suggest giving it a meaningful name, so you know later what that pcap is without having to run it.

Make a sub-directory, again, whatever you wish to call it, to store your indices in.
Now you need to index the pcap. The syntax is: xtractr index (index_directory)  --mode (basic|forensics) (pcap-file).

So if we had a pcap called "dns-traffic.2.4.11.pcap" and a sub-directory called "index_dns", and we wanted full data (forensic), we would run: xtractr index index_dns --mode forensics dns-traffic.2.4.11.pcap . Depending on how big your pcap is, this might take a little while to run, xtractr will give you a progress meter while running and return to prompt when down. You can omit the mode parameter, by the way, and xtractr will default to basic.
 (Continued - Blogger is not co-operating today)