BPF Filters Across Protocol Header Boundaries

BPF has keywords to address layer 3 and layer 4 protocols, such as IP, TCP, UDP and ICMP. But what if you need check the value of a field in a higher layer protocol, such as DNS? If the header you're working with is static, that is, never changes size like UDP, you can then go past the end of that header and into the next. Johnathan Ham teaches this in the excellent FOR558 Network Forensics class at SANS.
To use his example, say you wanted to filter packets to see only DNS queries. The QR bit in the DNS header, which specifies whether a DNS packet is a query or an answer, is found in the 2nd byte offset from zero in the DNS header, or the third byte. Since a UDP header is always exactly eight bytes long, we can extend what byte we specify past the end of that header and into the DNS header. The last UDP byte is number 7 (it consists of bytes 0-7), and we want to drill down to the second byte from zero in the DNS header. Counting from that 7th byte, that would be the 10th byte from the beginning of UDP. Our field is one byte in length, and the QR bit is first bit on the left, or as it's known, the highest order (or most significant) bit. Networking is big endian, meaning we address bits from the left to the right, the same way English speakers read. Now we need to apply our bit masking to check the value of just that one bit. As we know, we want to apply a bitwise AND operation to all the bits, using a one to preserve the bit and a zero to mask the bits we don't care about. So our byte is laid out like this:

QR  Opcode   AA TC RD
 0     1  2 3 |4   5    6    7

Our first nibble, or four bits, contains the QR bit and the first three bits of the Opcode field (bits 0-3).
Our second nibble contains the last Opcode bit, the AA bit, the TC bit and the RD bit (bits 4-7).

If you want to know what those other bits are used for, you can take a look here.

So our mask would be 1000 0000 in binary. Our first nibble has a one in the 8's column (2 to the power of 3) and everything else is zeroed out, so in hex our mask would be 0x80. We apply that mask using our BPF filter and look for a value of 0 in that bit (which designates a query, as a value of 1 designates a response).

We can add port 53 for some extra assurance we get DNS queries and not syslog or other UDP traffic. The tcpdump command I used is: tcpdump -nnvvv -i eth0 'udp[10] & 0x80 = 0 and port 53'

And sure enough, what I end up is packets that look something like this:


21:20:49.596637 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 60)
    x.x.x.x.2051 > 208.67.222.123.53: [udp sum ok] 2800+ A? ws.immunet.com. (32)

Here we see a query from one of my boxes (munged to x.x.x.x) to an OpenDNS server for the address record of ws.immunet.com.

Now if we change the value from 0 to our mask value in our filter (udp[10] & 0x80 = 0x80), we can look for DNS answers instead. (Or we could just specify not 0, since the only other possible value would be a one). So we could use 'udp[10] & 0x80 != 0'
Either one would work. Very nice!

NetSec and Linux

   Network Security requires having knowledge in a large number of areas. I can't think of a job in IT that requires a person to have at least some expertise in so many areas. How much a learning curve it is depends, of course, on where you came from to get to your first NetSec position. If you're starting straight out of college, and you took an Information Security track in school, you've no doubt been introduced to some or most of what you'll need. If you were already in IT, you may have moved into the field from a server or infrastructure team, or perhaps from desktop or a help desk. Regardless of what your knowledge base is, if you don't know the basics of working in Linux, you'll soon find out it's a prerequisite. (If you're going to work in Information Security and deal with things like access controls, policies, audits and vulnerability testing, you may not need much in the way of knowing Linux.)

   The primary reason you'll need to get familiar with Linux is that the majority of NetSec applications (like IDS/IPS, log servers, packet auditing and so forth) and security tools run either exclusively or best on Linux.
There are some very decent Windows-only tools and some popular tools have Windows ports of them, but most of the really good tools were designed to run in Linux. (I've been told that Wireshark is a rare exception and actually runs better on Windows than Linux, but I have no proof that's the case).
Fortunately, there is an absolute glut of sites that's sole purpose is to help you learn Linux step by step. And Linux has a huge user base that relishes in helping out people new to the OS and getting them up to speed. You can take commercial, paid training courses if you wish, at places like New Horizons, Babbage-Simmel or other training centers, but the help you get through the Linux community makes that unnecessary (though if your company WANTS to pay you to learn Linux for a week in a classroom, no reason to turn it down).
Linux is, of course, free for the overwhelming majority of the distributions (a rare exception would be Red Hat Enterprise, where you pay for support and updates, but there's also the free community supported Fedora or CentOS which operationally are almost identical). You can download and install Linux on a box and get started immediately, or make a live boot disk on a CD, DVD or flash drive and use it anywhere. You can install the (free) virtual environment VirtualBox (from Oracle), then download several distros you'd like to try and run each one as a VM.

   However you  do it, you'll want to get up to speed as quickly as possible, as you'll find each new tool you need to do your job probably only runs on Linux or runs best on it. Linux has come a long way in terms of GUI support, but one of the advantages of Linux over Windows is how much less overhead it takes, so running from the terminal instead of X Server (the Linux GUI) is probably best. And you might as well learn to work from the command line from the start, as there may be situations where you'll need to work on a box that doesn't have X installed.

   As to what flavor to run, you'll not find anything close to a consensus anywhere. Everyone has their favorite, and most are passionate about why they believe their distro is superior to others. Personally I like Fedora and I managed boxes running RedHat Enterprise for many years. The boxes I inherited at my current position run another flavor (though now that I've taken over the care and feeding, that will change at the next reload). I've worked with Debian, Ubuntu, Slackware and others and most of the commands are identical except in things like network setup and package management.Other popular distros include CentOS, Mint, Mandriva, Arch, and Gentoo. Those are just a few. The site distrowatch.com is a great place to get the newest versions of whatever you run and find something new to try out. There are currently 317 distributions of Linux listed on the site (note that distrowatch does not host all these versions, but provides info on them and links to the developers site where you can download them).

    If all you've ever used before is Windows, there IS going to be a learning curve. But you might as well get started, because to work in NetSec, you'll need to learn a second "language". In my experience, the person who advised me told me to download FreeBSD and learn it (a flavor of UNIX, not Linux) as my first non-Windows OS. That was a bit of a nightmare and definitely not recommended as a first new system to learn. I know any version of Linux will be much kinder to you than that (though FreeBSD is a great system and very secure).  Have fun.

Multiple Byte BPF's

If you're just beginning to use BPF's, you'll soon find you need to address fields than span more than one byte. This is very easy to do with a BPF (Berkeley Packet Filter, if you didn't read the previous post). We know we begin our filter with the protocol header we want to work with, such as tcp, ip, udp or icmp and specify the byte the field is in (starting with that byte offset, or counting from zero.) When we worked with the TCP flags field, we used tcp[13]. When we need to work with a field that spans multiple bytes, all we need to do is append a colon to that byte number and then specify how many bytes we want the filter to read. So if we wish to look at the source port field, we would write it as tcp[0:2] (the source port field resides in byte 0 and byte 1 of the TCP header). Now our filter will look at just those two bytes. To only look at packets who source port is 37, we would use the filter 'tcp[0:2] = 37'
BPF, however, has built in primitives, or keywords that apply filters, including "src" or source and "port". So if we wanted to see these packets, the easier way would be to use 'src port 37'.
Other fields have no primitive equivalent. If we needed to see all packets with a TTL greater than 64, we would specify that with a filter like this: 'ip[8:2] > 64'. We're telling tcpdump we want to start at the 8th byte offset from 0 and evaluate two bytes and show any packets that have a value in that field greater than 64.
With your protocol header charts in front of you, you can create a filter to show you any byte in any header and check it against whatever value you need.
By the way, if you're capturing packets for monitoring or auditing purposes, it's usually best to use the largest snaplength (size of each packet) that your storage space and processing power allows, and then use your filters to pull out the packets you need later.

BPF's and Bit Masking

BPF's (Berkeley Packet Filters) allow you filter down to the bit, not just the byte, of a protocol header. There a number of fields that might require you to do this, most notably, the TCP or IP flags fields.

The TCP flags indicate what type of packet it is. They are:

SYN - synchronize or begin the connection. SYN packets are only used during the initial three way TCP handshake.
ACK - acknowledgement that a packet was received
FIN - finish the connection gracefully with both sides shutting it down in an ordered manner.
RST - reset tears down the connection with no further exchange of packets, usually used when a host denies an attempted connection or a the client is terminated abruptly, like closing a web browser.
PSH - the push flag indicates to the other connection that it should process this data as soon as possible  instead of queuing it in the TCP buffer
URG - the urget flag is used to process data that must be processed immediately (such as a character in a telnet session) and uses a pointer to indicate the last byte of urgent data.

These fields are located in the TCP header at the thirteenth byte offset from zero (protocol fields start at 0, not 1). Proceeding them in this byte (highest order bits, or left-most since networking is big endian) are two flags used for differentiated services, or what used to be known as ToS (type of service). They aren't related, just know they are there as it will matter when we construct a BPF.

So our bits are laid out like this:

C E U A      P R S F

In the first nibble (4 bits) we have the two differentiated services flags, then the urgent flag and finally the acknowledgement flag.

In the second nibble we have the push flag, the reset flag, the syn flag and end with the fin flag.

So if we need to see only one type of packets to do our analysis, we need to employ bit masking to narrow our BPF down to that bit or bits. The way we do this is to write out the entire byte and mask out the bits we don't want to see and retain the bits we do. To do this, we apply a zero to bits we want to filter and a one to bits we wish to retain. For example, to see only SYN-ACK packets:

C     E     U     A     P     R     S    F
 0     0     0      1     0      0     1    0

We have zeroed out all the bits except the ones corresponding to the ACK and SYN flag. The ACK flag is the least most significant bit in the first nibble (4 bits make a nibble and two nibbles make a byte) and the SYN flag is at the second least significant bit in the second nibble.
This would give us a binary mask of 0001 0010. Convert that to hex and we have 0x12. To apply this as a BPF filter, we do a bitwise AND  operation on the byte. Any 1 ANDed with another 1 equals one, and a 1 with a 0, OR if both are 0, equals zero.

We use the "&" operator to do the bitwise operation. In BPF filters, the byte number is appended in square brackets to the protocol header we're working with. So in this case, we're working with the TCP header and byte 13. So the first part of our filter is tcp[13].

Now we add the bit shifting. So the second part of our filter is: "& 0x12". And finally, we tell it what value to compare the result to. Since we are checking for SYN-ACK packets, we want the entire byte to equal 0x12, which would be the value if only the SYN and ACK flags were set. So our entire filter would be:

'tcp[13] & 0x12 = 18' or we could write it 'tcp[13] & 0x12 = 0x12' since 12 in hex equals 18 in decimal. Either way would work.

You should always put your BPF in single quotes (on Linux boxes) or double quotes using WinDump, as a good practice. Your BPF may not be complex enough to need it every time, but if you always do it, it won't bite you when you have a compound filter that requires it.

Using this method, we can now specify any bit or bits in any byte in any header. Very powerful tool.

If you're new to packet analysis, you may need to learn or do a refresher on hex and binary, and look at bitwise operations, but that, with a copy of your protocol header charts, is really all you need to accomplish this.

You can span multiple bytes, chain multiple filters together, mix primitives (built in filters, like tcp) and complex filters, and get as specific as you need. And you can save your very exotic filters into a file and have tcpdump read them in from the command line with the -F switch.