(Photo: SANS FOR572 Courseware. Phil Hagen, author)
(Photo: SANS FOR572 Courseware. Phil Hagen, author)
One of the most powerful aspects of nmap are the scripts written to extend it's functionality. These scripts have an .nse extension, meaning they are part of the Nmap Scripting Engine. The default location for nmap scripts is at /usr/share/nmap/scripts.
Using them you can do a rather thorough vulnerability assessment with nothing more than nmap. The scripts are prepended with a classification, shuch as http- or dns- or smb-, which makes it easy to search for, or use bash's auto-complete. But, you can use an entire category of scripts with a wildcard. To use one script, use --script=<script_name>. If the script takes any arguments, use --script-args=<argument1,argument2>. To use an entire class of scripts, wildcard the parameter such as --script=http*. Youy can update the scripts anytime with the command nmap --script-update.
Wireshark is a nice tool for keeping your packet dissecting knowledge fresh and finding things you didn't know were there...you can play with the colorization settings and click through the fields in the packet dump and see which fields in the header and payloads they correspond to. Below I have the start of the IP header selected, which shows us the first nibble is the IP version field, and the other half of the first byte is the header length. We see the IP version is 4. Whatever is in the second nibble, the IP Header Length, multiply it by four to get the bytes in the header. So the value five means the header is twenty bytes, the minimum length of an IP header, which also means there are no IP options. IP options are (relatively) rare and almost always suspicious if not malicious, whereas TCP options are common, with things like padding, selective acknowledgment, etc being represented.
Note: In my haste, I switched the value and multiplier in the second nibble when I originally posted this. Sorry..
Nmap Basics Part 2
Nmap can take a variety of different target ranges. CIDR notation is still supported as in 188.8.131.52/24.
You can also specify a custom range, like 12.302.1-64.
You can mix and match specifications, so 184.108.40.206/24 220.127.116.11-128 would be valid as well.
You can also feed a file to nmap with a mix of IPs, ranges, and hostnames.
Entries must be separated by spaces, tabs or newlines. I suggest the common one entry per line format for easy readability, for anyone looking at the scan and needing to see what the scope was.
nmap has 6 built-in levels of scanning intensity, from Paranoid, -T0, to Insane, -T5. The default is -T3. Using -T4 should be acceptable if you have a good network connection between you and the target.
-T0 and -T1 are very stealthy and therefore very, very slow. Seeing most organizations don't have the bandwidth to even investigate port scanning, this level of IDS evasion probably isn't worth using. Running your scan through Tor is also usually effective unless the target has geo-blocking in place for whatever exit node you come from. -T5 is almost certain to get you noticed and unless your connection is very solid, runs the risk of dropped packets.
Nmap 101 Tutorial
There are two common types of scans, the syn scan and the full connect scan.
-sS - Syn scan sends a syn packet, and if it receives a syn-ack, marks the port as open and sends a reset and tears down the session. Must be root to run a syn scan, as it manipulates the network stack to send the reset out of sequence.
-sT - Connect scan. Completes the full TCP/IP handshake. Any user can run a connect scan as the network stack is not manipulated.
-v verbose mode. Can be repeated for increased verbosity, will update the status more often. If -v is not used, using the space bar you can force a progress update of the scan.
--open Only report on ports that were found open at the end of scan summary.
-sV - version scan. Reports on the version of platforms of running services like http, ssh, etc. Varying degrees of effort to determine the version, use --version-all for the most accurate results at the cost of a slower scan
-O OS scan - attempts to determine the operating system of the host being scanned. Note both version and OS scans use scripts written for nmap written in NSE, Nmap Scripting Engine that extend functionality. Use --osscan-guess for the most verbose results as it will fuzz the scan, again at the cost of a slower scan.
-e Show the link later headers in addition to higher-level protocol headers.
-x hexadecimal output.
-X hexadecimal and ASCII output. Common when there's ASCII text in the packet than can be displayed.
-A show only ASCII data.
Output can be saved in several formats. We will go into details later. Use -oA to save in all formats in separate files, including greppable (deprecated) and XML.
-Pn - don't ping the hosts before port scanning. Useful on networks that block ICMP echo requests.
What are Berkeley Packet Filters? BPF’s are a raw (protocol independent) socket interface to the data link layer that allows filtering of packets in a very granular fashion1.
BPFs were first introduced in 1990 by Steven McCanne of Lawrence Berkeley Laboratory, according to the FreeBSD man page on bpf2.
Working with BPF
If you use tcpdump for very long, you encounter what are called “primitives”, filter expressions to tune your results to only see certain traffic. Examples of primitives are “net”, “port” “addr” and qualifiers to those such as “src” or “dst”.
With these, we can limit our results using filters such as ‘src host 10.10.1.1’ or ‘net 10.10’. There are many of these (see the man page of tcpdump for the full list)
You can also specify protocols, such as “ip”, “tcp”, or “icmp”. Some even make comparisons, such as “less” and “greater” for packet length.
These primitives are shortcuts for BPF’s. Each one references some field or fields in one of the network protocol headers. For example, the embedded protocol field in the IP header is the 9th byte offset from 0. If the value contained there is a 6, the packet is TCP. So the primitive “tcp” really means show me all the packets in the IP header whose 9th byte offset from 0 contains a 6. If we wrote this as a BPF, it would look like this: ‘ip = 6’ or using hex, ‘ip = 0x06’ .
BPF’s can go far beyond the built-in primitives, allowing us to get as granular as needed, down the single bit level. If a field does not span the entire byte, we’ll need to write a BPF to look at the bits in question to determine the value there.
Let’s look at the first line of the IP header3 to see an example.
IP Header length
Type of Service
We see byte 0 (we start counting from 0, which is what we mean by offset from 0) that there are two fields in the byte, the IP Version field, and the IP Header Length Field.
If we wanted to see what the IP version of the packet is, how we would do this? We only want the value in the high order nibble (high order = left most as we count bits from right to left, and a nibble is 4 bits, or half a byte). To see that value we have to extract it from the byte of data somehow and look at it singularly. To do this, we employ a method known as bitmasking. Bitmasking is simply filtering out the bits we don’t wish to look at and retaining the ones we do.
To accomplish this, we’ll perform a bitwise AND operation on all of the bits in the byte. If we AND the bits, only the ones with a value of 1 will be retained. Let’s look at this.
Here’s a binary representation of a typical first byte in the IP header:
0 1 0 0 0 1 0 1
We’ve separated the two nibbles here for clarity. We see the low order nibble (right-most) has 0101. This is our IP header length. We want to check the high order nibble, which has the value 0100. To do this we will add 1 to each bit. In a bitwise AND, any values except two 1’s equal 0. Two 1’s equal one.
So to manipulate the bits to see the first nibble only, we want to add 1’s to the high order nibble and 0’s to the lower order. Since all 1’s will equal F in hex, we will write an expression adding hex F to the first nibble and 0 to the second.
Here’s what the BPF will look like:
ip & 0xF0 = 0x04 (our search value).
Broken down, we are telling tcpdump to look at the IP header (ip), first byte offset from 0 (  ), retain all the bits in the first nibble and discard all the bits in the low order nibble ( & 0xF0 ) and show us all the packets with a value of 4 in that nibble ( = 4).
Here’s our bit-wise operation…
0 1 0 0 0 1 0 1
0 1 0 0 0 0 0 0
We now see the low order nibble has been filtered (all 0’s) and we have the high order nibble left. Binary 0100 = decimal 4, so this shows us the packet has value of 4 in the high order nibble of the first byte; the IP header is set to IPv4.
Now that we see how BPF’s work, here are some samples of filters we can search on:
'ip = 0x11' udp
'ip = 0x01' icmp
'tcp[2:2]' 2nd byte, spanning two bytes
'icmp = 0x08' echo request packet
'tcp[2:2] < 0x14' tcp dest port < 20
Let’s create a filter for one of the more common and more complex uses: TCP Flags
The flags field in TCP is found at the 13th byte offset from 0. The flags themselves inhabit all of the lower order nibble, and the two lower-order bits of the high order nibble.
The two high-order bits of the high-order nibble are used for ECN (Explicit Congestion Notification). Here’s our layout…
TCP Byte 13
Flags CWR ECE Urg Ack Push Reset Syn Fin
Binary Values 128 64 32 16 8 4 2 1
Let’s assume we wish to see all packets with the SYN and FIN flags set. This is anomalous behavior and is usually indicative of a port scanning method.
We would need to look at the whole byte and retain all bits except the two highest orders in the high order nibble. To do this we need a mask retaining all of the lower order nibble and the lower order bits of the high order nibble. Here’s our bitwise operation mask:
High order Low order
CWR ECE Urg Ack -- Push Reset Syn Fin
0 0 1 1 -- 1 1 1 1
So, we would have a hex F in the low order nibble (all 1’s), and a 3 in the high order nibble (a 1 in the 0 column, which is 1, and a one in the 2’s column, which is 2, equals 3).
So our mask would be 0x3F. That would show us only the bits that contain TCP flags.
If we use that filter and look for a value of 3, meaning the two lowest order bits are set, the Fin and Syn bit, we would end up with this:
tcp & 0x3f = 0x03’
This filter tells the system to filter on the 13th byte offset from 0, discarding the two highest order bits, and showing packets the have a total value of 3 in the six remaining bits, which would mean the Fin and the Syn flags were both flipped on.
Now that we know how to look at only the bits we need, we can apply this to any field, in any network header. You can, of course, string multiple filters together to get as specific as needed. Here’s a tcpdump query to show us all packets with the Syn flag set, and a datagram (packet) size greater than 134 bytes (probable data on the Syn packet), and an IP version that is NOT 4:
tcpdump –nn –i eth0 ‘tcp & 0x02 = 2 and ip[2:2] > 0x86 and ip & 0xF0 != 0x40’
Berkeley Packet Filters are a powerful tool for intrusion detection analysis. Using them will allow the analyst to quickly drill down to the specific packets he/she needs to see and reduce large packet captures down to the essentials. Even a basic knowledge of how to use them will save hours of time during the investigation of packets, or give insight into malicious traffic that wasn’t detected using other methods.
Wild West Hackin' Fest - Deadwood: I so love this conference. It's so much like DerbyCon. There aren't any "I hacked a car/plane/drone/coffee maker" talks. There aren't 10,000 people there and it won't make the national news like BlackHat or Defcon, but what it will do is give you the information you can use in your day-to-day job. Things to actually make your network safer. Flash and glitter are great and lots of fun. Huge parties and massages courtesy of IoActive at the Oasis are cool too if that's what you like. Less interesting but much more relevant is information you can apply to your job, day one after the con, and make your network just a little more secure.
In computer science, speed is all. Everything we do, we want to do as fast as possible. Right? No. Consider slow hashing algorithms. Some hashes are intentionally computationally expensive. Why? Because attackers have multi-GPU dedicated password cracking machines that can try millions of variations per second. A hash that requires more computation power is much slower to crack because of the time it takes to try each variation. Sure, your user may have to wait for half a second longer for the authentication form to go away and his page to load, but his/her credentials are many times more resistant to breach if an attacker obtains the hashed passwords. Consider this. On a fast, multi-GPU cracking machine, as many as 180 billion password hashes could be cracked per second. With bcrypt, that number would fall to 71 thousand hashes per second. Which one would you want YOUR password encrypted with?
Ten to fifteen years ago, a company having FPC (full packet capture) was an indicator of the seriousness of the company's information security efforts. Having trained analysts that could use those packets to analyze alerts from NSM devices was an even better indicator.
Today, the network landscape has changed to the point of having little similarity to a decade ago. The workforce was already starting to go mobile before COVID, and the pandemic forced a large swath of workers home. Mobile devices used for work became common and the services in the cloud, prevalent.
We started seeing diminished value in some of our traditional tools. Endpoint Detection and Response suites gave us better fidelity about what was happening on our endpoints, regardless of whether the device was sitting on the corporate network, at home, or at an airport or coffee shop. AntiVirus became less useful. Yet, it still had value as a layer of defense and was good for detecting common, prevalent threats. The fact that its effectiveness was diminished didn't mean the tool was no longer useful.
Packet captures are similar, in that though their effectiveness has been diminished by the use of SSL encrypted traffic, especially ECDH, it still has its place in layered defense. There is still much HTTP traffic on the Internet, plain text protocols are still in use even though safer alternatives have been prevalent for years, and they're a lot of cases where an attacker will use whatever means to do call-outs back to the attacking machine that uses unencrypted traffic. But there's another reason full packet capture is useful.
Consider the scenario where a host machine just made a connection to a known malicious website. The connection was made over TLS 1.2. You have packets from the connection, but the data is encrypted. An examination of the packets shows the TCP 3 way handshake, and five additional packets. In the first packet, there is some clear text obviously part of a certificate from the destination host, followed by three very small packets between the two hosts and finally a reset packet.
We can tell from the packet capture what happened, even though the data was encrypted. The client established a session with the site, the TLS handshake took place, and the session was ended. The rest was probably spoofed by a network security monitoring device if the traffic was not simply dropped by a firewall/IPS. What we can know is that there was no data exchanged between the two hosts.
If we have a full packet capture system set up, we can validate no data was exfiltrated, and no malware was downloaded, in seconds. Still very much valuable. And if there is no FPC solution, still worth the cost and effort to implement. (See Akime, formerly Moloch).
From here, with header diagram in hand, you should know be able to look at a packet dump and find the value you need, by counting from offset 0 to the correct field.
Some of the things you'll want to look for (like IPv6 traffic or only ICMP traffic or a particular IP) can be done most efficiently with primitives.
Others, like, options in the IP header, which are rare and always suspicious can be done by by checking the header length field, but since the first byte in the IP header is broken into two fields of a nibble each, how do see just the header length, and not the IP version?
The answer is bitmasking. Bitmasking is hiding the bits we don't want to see and leaving the ones we do, simply put. For example, let's take the 13th byte (offset from zero, of course) in the TCP header.
Flags CWR ECE Urg Ack Push Reset Syn Fin
Binary Values 8 4 2 1 8 4 2 1
There's more to a primitive under the surface, and once we discover what it's actually doing, it opens up a whole new way for us to inspect and filter packets.
Lets use the TCP primitive as an example. What is it actually doing? How does it show us only TCP packets?
To see this we need to look at the IP header..not the TCP header. Here's a really simple version of a nicely color coded diagram.
There are many pcap tools available and which ones you use really depends on what you're using them for. Some are very good at just giving you the raw data, others parse the data and show you certain types of packets..
But maybe we should back up one step and define what we're taking about. What is a pcap? Simply put, a pcap is a binary file that contains packets captured off of a network interface.
How much data, and which fields depends on the manner that you capture the packets. We'll look at some of those options as look at tcpdump. I won't spend a lot of time on that tool because many people are familiar with it, at least the basics. It's actually a powerful tool when combined with BPFs, but we'll cover that as well later.
BPF is short for Berkeley Packet Filters and they allow pcap tools to get granular down to the bit, the BIT, level in specifying what header fields you want to see. For example, the TCP flags are each one bit in size. With BPFs you can specify to your tool to only show you packets that the SYN flag is set. Or SYN and any other combination of flags
The simplest BPFs are built into tcpdump. "tcp" and "port" are primitives. Others include udp, src dst, icmp, host and ether and net.
Other tools built on the libpcap library can use them as well, tools like ngrep and tshark.
An example would be tcpdump -nn -i eth0 'tcp and port 23'
This would tell tcpdump to listen on the eth0 interface, don't resolve hostnames, don't do port resolution, and only show packets that are the TCP protocol and either the source or destination port is 23.
If we only wanted to see traffic going to port 23, we could add the primitive dst in front of port.
tcpdump -nn -i eth0 'tcp and dst port 23'
Get into the habit of putting your BPF's into single quotes.
It will make no difference using simple BPFs like "port 80 or port 443", but when you get into complex BPFs using bitmasking, it will keep you from getting syntax errors.
So why use the -nn options, that disable name and port resolution? There are several reasons. One reason is speed and efficiency.
If your sniffing a fast, busy segment and have to do a DNS lookup on every address, it will slow tcpdump down to the point of starting to drop packets.
Another reason is if you're monitoring malicious traffic and the attacker controls his/her nameserver, they will see the DNS lookup and know they're being monitored by you.
Port resolution is done using Linux's services file, in /etc/. This is a mapping of ports to services, so when tcpdump sees port 80 it substitutes http for 80.
But can other services be bound to port 80? If you're root you can bind any service you like to any port you want. That port 80 traffic could be any protocol. So we'll bypass the commonly used ports and see what the traffic is ourselves.
1. Be a student of (information security, network security, cyber security). Always strive to know what the latest tactics, trends, and tools are and implement that knowledge into Operations.
2. Never allow a vendor to define what your greatest risk is. It will undoubtably be something their product detects well but be a low risk to your environment.
3. Be passionate about what you do. If you're looking for a 8 to 5 job you can forget about when you go home, information security may not be for you, to paraphrase Lesley Carhart.
4. Be a an advocate for what your company really needs as far as tools. There is a marked over-reliance on tools in the industry and not nearly enough emphasis on training and development, but you know from experience what tools you truly need.
5. Build the case, make the pitch, show verifiable results if you can. If you lose the battle, find an open source tool that does the same functionality without all the fancy reporting. Those reports aren't for you anyways; they're for the C-suite.
6. Cross train as often as you can. Learn as much as you can about penetration testing tools, because hackers and Red Teamers alike use some of the same tools. Using a tool will help you learn what to look for.
7. Always push for more logs, more packets, more flow data. Anywhere and everywhere you can. Your dataset is your bread and butter; the more complete it is, the better chance you'll have in detecting or even preventing an attack.
8. Don't dwell on defeats. Learn from it and move on. You'll see plenty of them, both from attackers and internally fighting for what you really need.
9. Ask to be present at every meeting, call, or demo with a vendor. Management can't always distinguish what is practical from the vendor hype and BS. Be the one to challenge their over-realistic claims and make them back them up with something more than carefully crafted demo data, tuned for maximum detection by their product.
10, Don't stay up nights worrying about nation states/intelligence agencies or highly funded, highly skilled APT entities. If they want in, they are going to get in. Your job is to make sure your company isn't the low hanging fruit.
11. To quote Mike Poor, prevention is optimal but detection is essential. Looking beyond the initial compromise and seeing what happened next is critical. Piecing together what the attacker did from disparate, seemingly unrelated pieces of information is something humans do better than machines, at least for now.
SANS has announced ALL of their Virtual Summits in 2021 will be free. See the link below for details..
Way to go SANS! That's giving back to the community..
If you're new to NetSec, you'll quickly find out network security is one of many silos in information security, and network security itself has many different roles. One of these roles is the intrusion analyst.
What exactly is an intrusion analyst? Duties can differ according to the size of the team you join, what type of platforms and tools are employed, but at the lowest common denominator, an intrusion analyst monitors security systems and investigates the alerts they produce.
This may encompass a SIEM, IPS, EDR, firewall logs and others.
SIEM: Security Information and Event Management
IPS: Intrusion Prevention System
EDR: Endpoint Detection and Response
Once an alert is received, triage begins. Many alerts can quickly be identified as false positives, generated from known benign processes. These are candidates for the continual process of tuning the SIEM and/or IPS. Without this process, the sheer number of FPs can begin to overwhelm the analyst. Automation has come a long way in doing triage for the analyst, and machine learning is helpful, but no system can replace the need for human inspection of many of the alerts.
Experience, tribal knowledge, awareness of changes in the network, applications used, integrations with vendors and business partners and again, experience, are skills ML or AI is not yet able to duplicate, if ever.
Is the intrusion analyst an incident responder? This again depends on the size of the team. Optimally, the intrusion analyst, after triaging an alert, would hand it off to an incident responder if action were necessary. This could anything from removing malware or re-imaging a machine to blocking IPs or applying isolation, either from the EDR solution or by network port shunning.
The reality is at most small and mid sized companies, the intrusion analyst will double as the/a incident responder, which is unfortunate. Not only does this require twice the amount of training and continual education, but it also means while the intrusion analyst is responding to an incident, he/she is not triaging alerts.
Sooty is a SOC analysts tool that would also work well for an individual intrusion analyst. It's available at https://github.com/TheresAFewConors/Sooty. I've installed it on a Fedora test box at my day job; I'm going to install it on one of my laptops at home and go through the steps here as I had to track down a couple of dependencies to get it installed correctly on Fedora 32.