IDS Signatures

I'll soon be moving into a new job, where I'll be supporting an IDS with a closed signature set. If you're new to NetSec and are tasked with researching out a companies first IDS/IPS, NOT buying a system with a closed signature set is a really good idea. What I mean by a closed set is that the signatures are not editable, or even viewable, by the analyst. Companies will claim "that's our proprietary information and we can't let anyone see it to prevent theft of our intellectual property".
Amazingly enough, other IDS companies have open signature sets and do not have any those predicted issues. What closing the signature set DOES do, to the analyst using the system, includes:

• Blinding the analyst to what traffic actually triggered the alert, making it difficult at best to determine if the alert was a false or true positive.
• Prevents the analyst from modifying the signature to narrow the scope to prevent too many false positives, or using it as the template for a new signature customized the company's individual needs.
• It keeps the analyst from sanity checking new signatures, to see if the vendor is putting out intelligent, well researched signatures, or just throwing out a very generic signature that will also trigger on hundreds of other packets with similar packet data. Companies will do this to allow them to say their signature base detects threat xyz, regardless of whether it's a well written signature or not.

One of the first questions you should ask any vendor (in my opinion) is whether or not they have an open signature set and can the analyst/admin modify or remove signatures (through the GUI, and not through a request process to the vendor), AND, can custom signatures be added.
Another thing to ask, if the signature set is open, is whether or not the signatures are written in a proprietary language or not. Being able to write signature by just following the prescribed format (like Snort or Dragon) is a great boon to the analyst, and cuts way down on the learning curve to get operational. Anyone can learn to write a Snort signature in a short time. However, learning to write a good Snort signature takes a good deal more investment of time..

IDS Placement

A discussion on the SANS Alumni Group on LinkedIn started up concerning IDS placement. I weighed in with this:


"I've had this discussion before when a company I worked for mandated ONE IDS at the ingress points, with additional ones optional. I've always believed behind the firewall would be where you would put the IDS if you could only have one. My reasoning is that 1) behind the firewall shows you what traffic just made it past your router ACL's, firewall rules, reverse proxying,etc. 2) behind the firewall lets you see the real IP of outbound traffic instead of the hide address, so when boxes get infected or compromised and start connecting out, you can shut down the traffic and easily trace the box to clean it. 3) Outside the firewall keeps down the white noise of junk your perimeter defenses and allows you to concentrate on alerts of interest. If you have both, you tune your external IDS for the traffic you need to see, like specific high value servers or boxes that attract a lot of attacks, to gauge what preventive shunning needs done.
Mike Poor talks about the umbrella sensor, that is the sensor right at the edge of the network that's supposed to detect all traffic to everything behind it and how it's not a good architecture. Ideally, you should have layers of sensors at strategic points, like in front of your web farm, in front of your name servers, in front of WAN links with partners and clients, etc and then you can tune each one appropriately with what signatures you run and with filtering. That's the ideal set up. For many, especially those smaller companies, that umbrella sensor is all you're going to get. In that case, behind the firewall is the only place to put it..."

What do you think (especially those new to the network security field)? Assuming you had to work with an umbrella sensor and could only have one, where would you put it?

Practice Makes Perfect

If you're just starting out in network security or working towards that end, packet practice will increase your familiarity and improve your speed in doing analysis. It's best to get better and faster now, when there's no pressure or duress, then when an incident is happening and folks are yelling for answers... or what you find determines whether a production system should be taken off-line.
By practice, I mean spending time working with packets. If you have monitoring access to your companies network, that would be the place to start. Not only will it give you access to a wider range of traffic than using your home network, but you'll start becoming familiar with what is normal. It's very difficult to find the anomalous if you don't know what's normal. If you don't have access to your companies traffic yet (and ONLY use your companies network if you have been given explicit permission, in writing, to do so), there are tons of places you can pull down interesting packet captures to play with and practice.  pcapr.net has a huge repository, the HoneyNet project has monthly challenges, the Army ITOC posts captures from the Inter-Service Academy Cyber Defense Competition (the NSA provides a Red Team against the the service academies Blue Teams), and so forth.
There all different kinds of ways to practice. You can do scenarios; choose an attribute that would indicate possibly malicious activity, then see if you can find any on your network. Do a realtime sniff for any packets whose IP header is greater than 20 bytes, indicating that IP options are in use. Look for large sized ICMP packets (greater than 100 bytes), and if you find some and they aren't malicious, determine why they're large (Microsoft is one culprit in this category).
You can redirect the captures into text, and use awk and sed and grep and other utilities to start building up a collection of scripts and filters for things you'll want to look for often. If you're a perl programmer, you can use all the nifty perl networking modules to make yourself some automated programs to break apart captures and extract data.
You can use tools like xtractr and NetWitness Investigator to see your high level categories, then drill down and look at the individual packets when you see something interesting. I've been using both to look at events from our IDS.
 You can run the packets through dnsiff and see if there are clear text authentication tokens you weren't aware of. Run them through ngrep and use keywords specific to your corporation and look for data leakage going beyond the perimeter of your network. You can load them in EtherApe and watch for the blooms of color.. readily identifying "top talkers", which could be indicative of malicious activity, a network issue or a bandwidth hog, like someone using BitTorrent.
There are dozens and dozens of tools you can run those captures through, and the more you do it, the more familiar you'll become with your network and the better at identifying those folks who refuse to play nice.
Links to packet captures:

• pcapr - http://www.pcapr.net/home
• HoneyNet - http://www.honeynet.org/challenges
• ITOC - http://www.itoc.usma.edu/research/dataset/ (bottom half of the page.. scroll... they're there)