Awking from Hal Pomeranz

On the Command Line Kung Fu blog (blog.commandlinekungfu.com), Hal Pomeranz has written up an excellent introduction to the awk utility. You can view it here. It's the clearest, easiest to understand article on awk I've ever seen. Kudos to him and Ed Skoudis for maintaining this excellent resource.

More Detailed Spondulas Example

One of the advantages of Spondulas over other similar tools is that it supports POST requests. Here's an example from an IDS alert I recently saw, showing the additional steps in submitting a Spondulas request using the POST method:


[root@muckabout spondulas]# python3 spondulas.py -u "http://www.newsday.com/logger/p.gif" -a -r POST
http://www.newsday.com/logger/p.gif
POST requests must have variables.
Please enter POST variables......

Format: parameter1=value&parameter2=value&parameter3=value....

Post vars: parameter1=a=1.4275134&parameter2=d=/2.810/2.893/2.906&parameter3=type=MIXEDTYPE&ct=430_432,430,0

Enter a referrer if you were redirected from another site.
If there is no referrer, you can leave this blank.

Referrer should be in the format: http://www.example.com/somepath/file.html

Referrer: http://www.newsday.com/sports/football/bears-wr-brandon-marshall-heard-of-players-using-viagra-to-gain-edge-1.4275134

Cookies are used to track state on the same web site.
Enter any cookies that were set for this web site...

Cookies should be in the format: cookie1=value1; cookie2=value2

Enter each line separately. Press enter on a blank line to finish entering

Cookies: visited=true
Cookies: JSESSIONID=222C0040D266FDD184C0FAD6E0065177
Cookies: SKSESSION=10.5.145.229
Cookies: s_pers=%20gpv_ch%3DSports%7C1354918554944%3B%20s_depth%3D1%7C1354918554944%3B%20s_vnum%3D1357508754944%2526vn%253D1%7C1357508754944%3B%20s_invisit%3Dtrue%7C1354918554944%3B%20dslv%3D1354916754944%7C1449524754944%3B%20dslv_s%3DFirst%2520Visit%7C1354918554944%3B%20s_vnum_w%3D1355029200959%2526vn%253D1%7C1355029200959%3B%20sinvisit_w%3Dtrue%7C1354918554959%3B%20ri_ch%3DSports%7C1355089554959%3B%20ri_c38%3Darticle%7C1355089554959%3B%20sspp%3D1354916754959%7C1354918554959%3B
Cookies: tmpPersistentst
Cookies:


Query being sent
----------------
POST /logger/p.gif HTTP/1.1
Host: www.newsday.com
User-Agent: Mozilla/4.0 (compatible; MSIE 6.1; Windows XP)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://www.newsday.com/sports/football/bears-wr-brandon-marshall-heard-of-players-using-viagra-to-gain-edge-1.4275134
Cookie: visited=true
Cookie: JSESSIONID=222C0040D266FDD184C0FAD6E0065177
Cookie: SKSESSION=10.5.145.229
Cookie: s_pers=%20gpv_ch%3DSports%7C1354918554944%3B%20s_depth%3D1%7C1354918554944%3B%20s_vnum%3D1357508754944%2526vn%253D1%7C1357508754944%3B%20s_invisit%3Dtrue%7C1354918554944%3B%20dslv%3D1354916754944%7C1449524754944%3B%20dslv_s%3DFirst%2520Visit%7C1354918554944%3B%20s_vnum_w%3D1355029200959%2526vn%253D1%7C1355029200959%3B%20sinvisit_w%3Dtrue%7C1354918554959%3B%20ri_ch%3DSports%7C1355089554959%3B%20ri_c38%3Darticle%7C1355089554959%3B%20sspp%3D1354916754959%7C1354918554959%3B
Cookie: tmpPersistentst
Content-Length: 97

parameter1=a=1.4275134&parameter2=d=/2.810/2.893/2.906&parameter3=type=MIXEDTYPE&ct=430_432,430,0



Do not be alarmed if the progam appears to "hang."
This is caused by keep-alive packets. A timeout exception
will be raised after 30 seconds.

Birds away.....
.IP address: 66.54.17.120
..............................
Target URL: www.newsday.com/logger/p.gif
IP address: 66.54.17.120
Referrer: http://www.newsday.com/sports/football/bears-wr-brandon-marshall-heard-of-players-using-viagra-to-gain-edge-1.4275134
Date/Time: 2012-12-13 11:09:05
Output File: 001.txt
Links File: 001-links.txt



Cookies
--------------------
visited=true; domain=www.newsday.com; expires=Friday, 14-Dec-2012 16:08:34 GMT; path=/
tmpPersistentstatsUserId=63c7555ac82755e12360f5da377883a4; Domain=.newsday.com; Expires=Fri, 13-Dec-2013 16:08:34 GMT; Path=/
SKSESSION=10.5.145.228; path=/

[root@muckabout spondulas]#

Another Handy CLI Tool: iftop

iftop is a handy way to quickly start a bandwidth monitor on an interface from the command line. It takes a number of parameters, most of which can be toggled on and off from the ncurses interface (like top).
-n disables hostname look ups and -N does the same for ports. -p runs promiscuous mode and -P turns on the ports display (appended to the end of the IP address and separated with a colon, like tcpdump). -b disables the bandwidth meter (a highlight bar on the IP's row that shows graphically how much traffic is being passed), and -B changes the stats to bytes per second instead of bits. -i specifies the interface, like many network tools and -f specifies the filter. You don't need to specify a filter of "ip" as only IP packets are counted; it's already set.
Other  parameters deal with netmasks and an optional alternate config file.