IDS Evasion Part I

Methodologies to avoid detection by IDS involve one of three types of techniques: evasion, insertion and denial of service. These are defined and described in the landmark paper "Insertion, Evasion, and Denial of Service:Eluding Network Intrusion Detection", released back in 1998 by Thomas Ptacek and Timothy Newsham. A new intrusion analyst would do well to read this paper, mostly still relevant after 13 years, and keep a copy for reference.
An insertion attack is where an IDS accepts a packet that the destination host rejects.
An evasion attack is where an IDS rejects a packet that the destination host will accept.

An example of an insertion attack would be where an IDS does not calculate TCP checksums, or does not calculate them correctly. An attacker sends a series of packets containing an exploit to host A. Within that stream of packets is one with a bad TCP checksum. If IDS B does not detect that the checksum failed, it will use that packet when it reassembles the stream and check it against it's signatures and processors. The match will fail because of the extra payload data contained in the bad packet, and the IDS will miss the attack.

Host A, however, will drop the packet with a bad checksum, reassemble the packets, and the exploit will be launched. The IDS never saw the attack. This is an unlikely scenario in 2011, but the next one I'll discuss involves at least one modern IDS that is still vulnerable to an evasion attack outlined over a decade ago.

NIC Offloading and Packet Capturing

Doug Burk, the guy who wrote and maintains the Security Onion Live DVD, posted a nice article on NIC offloading and the issues it can cause capturing network traffic here. Rather than try to regurgitate Doug's article, I'll summarize by saying it deals with the ability of modern network cards to offload the processing of traffic from the CPU, which is a great thing on fast (Gigabit) networks, but not what you want when capturing traffic. Why not? Because the controller does it's own reassembly at sizes greater than the MTU of your link, causing the sensor or sniffing program to miss part of the packet, in some cases, substantial parts of it. Doug is looking for feedback on this, so if you test and find you're getting "super-sized" packets in your packet captures (Wireshark will flag this for you as "Packet size limited during capture", let him know if you can.

IDS\IPS 101-6

Once your sensors are filtered and tuned, what you should have left are EOI's (events of interest), events that are either obviously malicious, suspicious, or enough of an anomaly to warrant investigation. How you investigate is dependent on a couple of factors. 1, What is your role? Or you strictly an intrusion analyst or do you do double duty as an incident responder as well? If  so, you'll need to go deeper in your analysis, possibly to the point of working directly on an affected server or desktop.
 But hopefully you're specialized as an intrusion analyst, and only touch alerts and the associated packets, and another team or person does the response and recovery.

Assuming that will be your role, what do you use to dig out the info you need to determine what happened?

Some IDS have a SIEM (security information event management) built in, like Enterasys' Dragon Defense, while others offer it as a separate, bolt on product (Symantec). Others are offered by SIEM-specific vendors such as Arcsight and LogRhythm. Log analysis tools like LogLogic and Splunk will also let you look at other data associated  with your alert.
But what if you work for a smaller company, and your IT security budget was just used  up with your IDS project? Fortunately there are lots of great open source tools you can implement to help you work through your alerts and determine what occurred (or didn't, as the case may more often be).

The first category of tools are the packet grabbers, the "packet auditing" tools. Your IDS will capture the packet or packets related to the alert, but not much else unless you specify it. Your IDS may not really be the place you want to collect more packets, as it's a busy box as it is. You can use another program on a separate host to do this for you, and do the analysis on that box as well, reducing the CPU cycles needed (it takes a while to decompress a 3 GB packet capture). I've mentioned many of these on this blog before, but I'll summarize them again here.

1. Daemonlogger (http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html) Written by Marty Roesch, author of snort, Daemonlogger will capture packets and write them out to a pcap file for you, or it can also act as a software-based tap, meaning it can capture on one interface and write back out to another. 

2. tcpdump - tcpdump has the ability to rotate pcap files based on size, doing the file naming for you, for x number of captures. Once you've reached the limit you specified, it will overwrite the oldest file with the newest. 

3. IDABench - based on SHADOW, IDABench is a perl based framework that captures packets into hourly pcap files, writes a summary of them out to a web page and allows you to search on multiple hours worth of captures using tcpdump, ethereal (yes it's old and outdated) and ngrep.Still a great tool. Contact me if you'd like a copy of it; it's no longer available anywhere on the Internet as far as I know.
Analysis Tools
1. Wireshark Wireshark (http://www.wireshark.org/) is one of the the most prolific protocol analyzers available. Wireshark supports hundreds of protocols and provides an interface that easily separates out the different headers to aid analysis, as well as allowing you to carve out content data.
2. Netwitness Investigator (http://netwitness.com/products-services/investigator) is a free form investigative tool that allows you to search and drill down into packets using contextual keywords instead of by header.
3. Network Miner (http://sourceforge.net/projects/networkminer/) Network Miner parses through pcap files or traffic on a network interface and identifies host names, operating systems, and open ports, as well as pulling out any files it identifies and filing them under directories for inspection.

IDS\IPS 101-5

Once you've determined what, where and how to deploy, and stand up your sensors, the real work begins. Once your IDS/IPS is up and running, the tuning phase begins (and never ends). If you're fortunate enough to be able to run the system in baseline mode for a couple of weeks to tune down the majority of the white noise, you can accomplish quite a bit using your trending or summary tools. Find the top talkers and start filtering out the obvious false positives that constitute large volumes of alerts. Lot's of internal processes can generate alerts that are benign, but look like events to the IDS, especially Windows networking processes. The IDS alerts themselves being forwarded to a mail server can even regenerate another alert for the same event, if you're including packet data (and the signature is especially broad in nature). Realize that once you've tuned your systems down to an acceptable level of alerts and feel confident you've about completed the process, you'll be doing this on a continual basis (or else see the white noise/false positive rate increase more and more over time). New signatures, additional network segments, new types of servers/services offered will all introduce FP's (false positives) into your system and will need filtered as they are identified. Your company may separate out the sysadmin duties from the analyst duties if the staff is large enough. For medium to small companies, you may wear both hats. I've found that's both an advantage and a disadvantage. The obvious disadvantage is that the more time you spend applying new signatures, filtering and tuning, and possibly even doing maintenance and patching, the less time you have to look at and analyze alerts. The advantage is that you, as the intrusion analyst, are uniquely qualified to make decisions or recommendations as to what traffic needs filtered, and what new signatures are relevant to your companies network infrastructure.

IDS\IPS 101-4

HIDS:
A HIDS (host based intrusion detection system) is installed directly on the server, and it's responsibility is to protect the system. The HIDS does not monitor traffic on the interface(s). Instead, it monitors it's application logs (such as Apache or IIS), and system logs (like the event logs on a Windows machine or syslog on a Linux box). Another important task of a HIDS is to check for changes to critical system files and alert when they have been modified (in the same manner as Tripwire). A HIDS will have a signature set, similar to the network IDS, that looks for strings associated with exploits being levied against the server. Since the packets reaching the host must be complete and formatted correctly, many of the evasion tactics used on the wire to bypass the NIDS will not be effective against the HIDS engine (such as fragmented packets with overlapping offsets, low TTL packets added to thwart the reassembly on the NIDS but dropped before they reach the destination, malformed packets for the same reason, etc.) A balanced, defense-in-depth security posture should include a HIDS on each public facing server at a minimum. Another good place to deploy host based detection is on critical systems such as database servers, human resources/payroll, servers that store PII (personally identifiable information) of clients or employees, and domain controllers and name servers.

IDS/IPS 101-3

Deployment:
One of the most critical decisions you will make concerning a new deployment is where to situate your sensors. Correctly locating your units will allow you advantageously monitor interesting traffic with less white noise to filter out. I was taught that there are two main schools of thought on the matter (and this is a 50,000 foot view of it):
1. The "umbrella sensor", as one of my SANS instructors called it. An umbrella sensor is one that sits at the periphery of your network, somewhere near your edge routers, firewalls and switches, and monitors all of the traffic coming in and going out. This may be fine (and the only financially allowable) solution for small, simple networks, but it doesn't scale well for complex networks. Large networks have many segments that need monitored, some public and some private, requiring many sensors. Encrypted tunnels such as VPN may make monitoring traffic impossible and infrastructure equipment on private segments with partners and clients may not be under your companies control. Sensors situated outside firewalls and routers may see so much white noise traffic as to make it nearly impossible to filter. If, however, your public infrastructure is small or budgetary limitations dictate you starting out with the "umbrella" approach, I would recommend placing your sensors behind your filtering equipment such as routers with ingress ACL's and firewalls. There are several reasons. The most obvious reason is noise. Unless you have near unlimited resources to look at and analyze events, you'll probably want to see just the traffic that made it past your router ACL's and firewall rules. Seeing what is traversing your network is critical and mandatory. Seeing what bounced off your hardened exterior is optional.
Another reason is that you'll be monitoring outbound traffic as well. Unless you're inside of any devices doing NAT/PAT such as firewalls, load balancer and proxies, you'll not see the real internal IP of an outbound connection. If that box was compromised and starts connecting back out to the attacker, and especially if data leakage is taking place, you'll need to shut that box down immediately. If what you are seeing is a NAT'd address, it may take you a while to find it in the logs (assuming you're logging on that device). With the real IP, you can locate and shut the offending device down easily (or isolate the port).
2. Targeted sensors. Targeted sensors sit close to the assets they are defending (like a Web farm). Because they are only a hop or two away, they see much less traffic (theoretically only traffic destined for one of those devices), so white noise is much less. The sensor can be tuned down to a smaller signature set specific to the protected assets. If your entire Web farm is Apache web servers running on RedHat, with no other services except what's needed for remote administration, your rulebase becomes a much smaller, more manageable subset of the whole. Hardware requirements drop as you're seeing much less bandwidth passing through the sensor, or the monitoring device attached to it. You'll probably want to still have a sensor behind the edge of your main ingress points as a first point of defense and a correlating packet collector, but your internal, targeted sensors will get most of your attention. If an attack has made it past all the filtering from the edge of your network down into one of your service segments, this would be a great time to see if your infrastructure blocking needs tightened up. IDS often is a validator of policy on your other security devices.

IDS/IPS 101-2

IDS vs IPS:
The difference between IDS and IPS is both in form and function. At a high level, an intrusion detection system is a passive device that monitors packets and identifies attacks using signatures and intelligence built into the IDS engine. The usual example is the string "get /etc/passwd", which used to mean (before shadowed password files) an attempt by someone to read the password file on a Linux/UNIX system. The IDS detects the string in the packets, and depending on how it's configured, alerts the analyst by some means, such as an email or via syslog or SNMP. This is a passive monitoring of the event (though some IDS have the ability to "shoot" down sessions by spoofing RST (reset) packets to both ends of the connection.) The IDS will have a management port with an IP address on it that allows it to forward it's data on to a collection/management console and allows remote access to it for administration (this may be done via an OOB (out of band) private network to limit access to the systems. It's monitoring port or ports will connect to a tap inserted at the desired point in the network, or a SPAN port on a switch.
Intrusion prevention systems sit inline, that is, they are connected directly to the network and all packets must pass through them. This is done via two NIC's (network interface cards), each attached to one end of the connection, such as between a firewall and a switch. Since all traffic passes through the IPS, like a router, the system has the ability to shun sessions it detects as being malicious by simply dropping the packets. No need to generate additional traffic in the form of reset packets. No notification is made to the attacker that the active response is taking place. The session, as the name implies, is prevented from continuing. Of course, if the event is a false positive, the very real possibility of dropping legitimate, business related traffic exists. Careful tuning and monitoring of the traffic must take place on an on-going basis to ensure the IPS doesn't interfere with normal traffic. Since the IPS sits inline, no tap or SPAN port is needed, but redundancy must be dealt with in case the device crashes or reboots. Most IPS are configured to fail open, which means if the software fails and stops working, the device will route all packets. Even then some traffic may be prohibited in the process of failing over. Some shops use a by-pass tap that routes all packets through the IPS as long as the interfaces are detected as being connected, but bypasses the device and routes the traffic directly if the interface goes down. Other use a second IPS, set up in fail over mode, so if the first device goes down, the second IPS takes over. The advantage here is twofold: you do not lose connectivity, the primary concern, AND you also do not lose your intrusion detection and prevention coverage while the first device is offline.

IDS/IPS 101-1

IDS (intrusion detection systems) and IPS (intrusion prevention systems) are a core part of network defense in-depth yet today, despite Gartner's proclamation that "IDS is dead" in 2003. The IDS/IPS market is still kicking, and for the intrusion analyst, most of the alerting he looks at will include events from IDS. That's even if they are aggregated in a SEIM (security event information manager) with firewall, router and host logs and alerting from anti-virus, web application firewalls and proxies.
If you're just getting started or have been tasked with standing up an IDS/IPS where none was previously, there are a number of things to take into consideration, even before you make your first contact with a vendor.
Some things to start working into you project plan would include:

1. Are you going to deploy NIDS (network intrusion detection systems), HIDS (host-based intrusion detection systems) or both, or are you going to deploy NIPS/HIPS (network or host-based intrusion prevention systems)
2. Where are you going to deploy your sensors? Depending on the budget being allocated, will you settle for a few "umbrella" sensors, boxes deployed along the perimeter of your network to try and see as many packets as possible? Or will you do targeted deployments, putting sensors in front of web farms, DNS servers, core switches, and critical internal assets such as database servers, payroll and HR?
3. How will your sensors monitor packets (this will be determined a lot by whether you go IDS or IPS). Will you use network taps or span ports for IDS? If you use inline IPS, how will you address redundancy? Does fail-open circuitry on your IPS box make you feel confident your phone won't ring at 3 AM when the sensor dies or reboots?
4.   How will IDS/IPS integrated into your network teams architectural plans? If the network topology changes, will you be able to adapt to things such as a change in media or major increases in bandwidth?
5. Who will monitor alerts? What will be your coverage window? If you will be the primary analyst, how is alerting covered during off-hours? Holidays and days off? If a team will handle the duties, is there coverage  if a member is out for an extended period of time? Are you considering an MSSP (managed security services provider) to handle the level one monitoring and alerting?
6. If this is a new addition to your corporate security infrastructure, will there be training provided for the analysts? (like the SANS Sec 503 Track). What about incident response? Will the intrusion analysts also be the incident handlers? If so, will there be training for that as well?