Friday, May 15, 2020

Quick Tip #4 - Dshell

Dshell is a network forensics tool that extracts data from network traffic, either from a pcap or live from a network interface.

You can us Git or Subversion to install the tool.

[root@legolas opt]# git clone https://github.com/USArmyResearchLab/Dshell.git
Cloning into 'Dshell'...
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 653 (delta 9), reused 17 (delta 7), pack-reused 624
Receiving objects: 100% (653/653), 924.42 KiB | 4.26 MiB/s, done.
Resolving deltas: 100% (292/292), done.
[root@legolas opt]# ls
Dshell  google
[root@legolas opt]#

Next is installing the dependencies.

There's no Fedora package for pypcap, so here I had to use pip for that one package...
[jeff@legolas opt]$ sudo pip install pypcap
[sudo] password for jeff:
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
WARNING: Running pip install with root privileges is generally not a good idea. Try pip install --user instead.
Collecting pypcap
  Using cached https://files.pythonhosted.org/packages/12/71/a689ca8c6ef4570e105efdc6b46d4ac16cdef7af36a8fbfaff0f6fa2b754/pypcap-1.2.3.tar.gz
Installing collected packages: pypcap
  Running setup.py install for pypcap ... done
Successfully installed pypcap-1.2.3
And install the rest of the prereqs...
[root@legolas opt]# dnf install geoip python2-dpkt  python2-IPy pycrypto
[root@legolas opt]# dnf install geoip python2-dpkt  python2-IPy pycrypto
Last metadata expiration check: 0:08:56 ago on Mon 23 Mar 2020 07:18:53 PM EDT.
Package GeoIP-1.6.12-5.fc30.x86_64 is already installed.
Package python2-crypto-2.6.1-25.fc30.x86_64 is already installed.
Dependencies resolved.
============================================================================================================================================================================
 Package                                     Architecture                          Version                                      Repository                             Size
============================================================================================================================================================================
Installing:
 python2-IPy                                 noarch                                0.81-24.fc30                                 fedora                                 37 k
 python2-dpkt                                noarch                                1.9.1-7.fc30                                 fedora                                261 k
Transaction Summary
============================================================================================================================================================================
Install  2 Packages
Total download size: 298 k
Installed size: 1.4 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): python2-IPy-0.81-24.fc30.noarch.rpm                                                                                                   86 kB/s |  37 kB     00:00
(2/2): python2-dpkt-1.9.1-7.fc30.noarch.rpm

Once the prereqs are satisfied, run make from the Dshell directory.
Once it successfully completes. run ./dshell. If you see a dshell command prompt, you're good to go..
[root@legolas Dshell]# ./dshell
root@legolas.net.att:/opt/Dshell Dshell>
Now run decode -l to get a list of available decoders..

root@legolas.net.att:/opt/Dshell Dshell> decode -l
  module                                   name                           author            desc
  ---------------------------------------- ------------------------------ ---------- --- -  --------------------------------------------------
  decoders.dhcp.dhcp                       dhcp                           dek        UDP    Extract client information from DHCP messages
  decoders.dns.dns                         dns                            bg/twp     TCP    extract and summarize DNS queries/responses (defaults: A,AAAA,CNAME,PTR records)
  decoders.dns.dns-asn                     dns-asn                        bg         TCP    identify AS of DNS A/AAAA record responses
  decoders.dns.dns-cc                      dns-cc                         bg         TCP    identify country code of DNS A/AAAA record responses
  decoders.dns.innuendo-dns                innuendo-dns                   primalsec  TCP    proof-of-concept detector for INNUENDO DNS channel
  decoders.dns.reservedips                 reservedips                    bg         TCP    identify DNS resolutions that fall into reserved ip space
  decoders.filter.asn-filter               asn-filter                     twp/nl     TCP +  filter connections on autonomous system number (ASN)
  decoders.filter.country                  country                        twp        TCP +  filter connections on geolocation (country code)
  decoders.filter.snort                    snort                          twp        RAW +  filter packets by snort rule
  decoders.filter.track                    track                          twp        TCP +  tracked activity recorder
  decoders.flows.large-flows               large-flows                    bg         TCP    display netflows that have at least 1MB transferred
  decoders.flows.long-flows                long-flows                     bg         TCP    display netflows that have a duration of at least 5mins
  decoders.flows.netflow                   netflow                        bg         TCP    generate netflow information from pcap
  decoders.flows.reverse-flow              reverse-flow                   me         TCP    Generate an alert if the client transmits more data than the server
  decoders.ftp.ftp                         ftp                            amm        TCP    ftp
  decoders.http.flash-detect               flash-detect                   ekilmer    TCP +  Detects successful Flash file download.
  decoders.http.httpdump                   httpdump                       amm        TCP    Dump useful information about HTTP sessions
  decoders.http.joomla-cve-2015-8562       joomla-cve-2015-8562           bg         TCP    detect and dissect malformed HTTP headers targeting Joomla
  decoders.http.ms15-034                   ms15-034                       bg         TCP    detect attempts to enumerate MS15-034 vulnerable IIS servers
  decoders.http.peht                       peht                           mm         TCP    Penetration/Exploit/Hijacking Tool detector
  decoders.http.rip-http                   rip-http                       bg/twp     TCP    rip files from HTTP traffic
  decoders.http.web                        web                            bg,twp     TCP    Improved version of web that tracks server response
  decoders.malware.emdivi.emdivi_c2        emdivi_c2                      bg         TCP    deobfuscate Emdivi http c2
  decoders.misc.followstream               followstream                   amm        TCP    Generates color-coded Screen/HTML output similar to Wireshark Follow Stream
  decoders.misc.grep                       grep                           amm        TCP +  Search for patterns in streams.
  decoders.misc.merge                      merge                          bg/twp     RAW +  dump all packets to single file
  decoders.misc.synrst                     synrst                         bg         RAW    detect failed attempts to connect (SYN followed by a RST/ACK)
  decoders.misc.writer                     writer                         twp        RAW    pcap/session writer
  decoders.misc.xor                        xor                            twp        TCP +  XOR an entire stream with a given single byte key
  decoders.nbns.nbns                       nbns                           dek        UDP    Extract client information from NBNS traffic
  decoders.protocol.bitcoin                bitcoin                        dek        TCP    Extract Bitcoin traffic, including Stratum mining protocol (pooled) traffic
  decoders.protocol.ether                  ether                          twp        RAW    raw ethernet capture decoder
  decoders.protocol.ip                     ip                             twp        RAW    IPv4/IPv6 decoder
  decoders.protocol.protocol               protocol                       bg         RAW    Identifies non-standard protocols (not tcp, udp or icmp)
  decoders.smb.psexec                      psexec                         amm        TCP    Extract command/response information from psexec over smb
  decoders.smb.rip-smb-uploads             rip-smb-uploads                bg         TCP    Extract files uploaded via SMB
  decoders.smb.smbfiles                    smbfiles                       amm        TCP    List files accessed via smb
  decoders.templates.PacketDecoder         unnamed                        xx         RAW
  decoders.templates.SessionDecoder        unnamed                        xx         TCP
  decoders.tftp.tftp                       tftp                           dev195     RAW    Find TFTP streams and, optionally, extract the files
  decoders.voip.rtp                        rtp                            mm         UDP    Real-time transport protocol (RTP) capture decoder
  decoders.voip.sip                        sip                            mm         UDP    Session Initiation Protocol (SIP) capture decoder
  root@legolas.net.att:/opt/Dshell Dshell>

Once in the Dshell prompt, use decode to start analysis, whether from a network interface or a pcap.
the syntax is decode -d <decoder> <source>
So to run the DNS extraction decoder from a network interface, the syntax would be decode -d dns -i <interface>
If you use tcpdump, ngrep or other libpcap tools, -i is easy to remember...

root@legolas.net.att:/opt/Dshell Dshell> decode -d dns -i eno1
dns 2020-03-24 15:57:02      192.168.1.1:53    --      192.168.1.4:44085 ** 34186 A? i.scdn.co / CNAME: scdnco.spotify.map.fastly.net, A: 151.101.46.248 (ttl 6s) **
dns 2020-03-24 15:57:02      192.168.1.1:53    --      192.168.1.4:44085 ** 38801 AAAA? i.scdn.co / CNAME: scdnco.spotify.map.fastly.net, AAAA: 2a04:4e42:b::760 (ttl 6s) **
dns 2020-03-24 15:57:10      192.168.1.1:53    --      192.168.1.4:36947 ** 9265 A? www.gstatic.com / A: 172.217.8.163 (ttl 156s) **
dns 2020-03-24 15:57:10      192.168.1.1:53    --      192.168.1.4:36947 ** 1339 AAAA? www.gstatic.com / AAAA: 2607:f8b0:4009:810::2003 (ttl 168s) **
dns 2020-03-24 15:57:10      192.168.1.1:53    --      192.168.1.4:43795 ** 3600 A? clientservices.googleapis.com / A: 172.217.6.3 (ttl 172s) **
dns 2020-03-24 15:57:10      192.168.1.1:53    --      192.168.1.4:58426 ** 18791 AAAA? www.googleapis.com / AAAA: 2607:f8b0:4009:816::200a (ttl 9s) **
dns 2020-03-24 15:57:10      192.168.1.1:53    --      192.168.1.4:43795 ** 33050 AAAA? clientservices.googleapis.com / AAAA: 2607:f8b0:4009:816::2003 (ttl 21s) **
dns 2020-03-24 15:57:10      192.168.1.1:53    --      192.168.1.4:58426 ** 9310 A? www.googleapis.com / A: 172.217.4.74 (ttl 156s) **
dns 2020-03-24 15:57:10      192.168.1.1:53    --      192.168.1.4:42267 ** 29350 A? accounts.google.com / A: 172.217.5.13 (ttl 109s) **
dns 2020-03-24 15:57:10      192.168.1.1:53    --      192.168.1.4:42267 ** 11695 AAAA? accounts.google.com / AAAA: 2607:f8b0:4009:806::200d (ttl 180s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:41977 ** 37896 A? www.google.com / A: 172.217.5.4 (ttl 24s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:41977 ** 1809 AAAA? www.google.com / AAAA: 2607:f8b0:4009:816::2004 (ttl 115s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:38453 ** 2485 AAAA? mirror.toolbar.netcraft.com / CNAME: d2nwnd7jypv9mn.cloudfront.net **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:38453 ** 22186 A? mirror.toolbar.netcraft.com / CNAME: d2nwnd7jypv9mn.cloudfront.net, A: 54.192.30.37 (ttl 60s), A: 54.192.30.3 (ttl 60s), A: 54.192.30.121 (ttl 60s), A: 54.192.30.32 (ttl 60s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:54375 ** 40826 AAAA? oauthaccountmanager.googleapis.com / AAAA: 2607:f8b0:4009:80e::200a (ttl 67s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:54375 ** 18283 A? oauthaccountmanager.googleapis.com / A: 172.217.1.42 (ttl 239s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:43085 ** 3682 A? www.gstatic.com / A: 172.217.8.163 (ttl 155s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:43085 ** 47980 AAAA? www.gstatic.com / AAAA: 2607:f8b0:4009:810::2003 (ttl 167s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:52599 ** 24895 A? fonts.gstatic.com / CNAME: gstaticadssl.l.google.com, A: 172.217.4.227 (ttl 151s) **
dns 2020-03-24 15:57:11      192.168.1.1:53    --      192.168.1.4:52599 ** 17736 AAAA? fonts.gstatic.com / CNAME: gstaticadssl.l.google.com, AAAA: 2607:f8b0:4009:813::2003 (ttl 184s) **
dns 2020-03-24 15:57:14      192.168.1.1:53    --      192.168.1.4:34639 ** 6929 A? clients2.google.com / CNAME: clients.l.google.com, A: 172.217.6.110 (ttl 200s) **
dns 2020-03-24 15:57:14      192.168.1.1:53    --      192.168.1.4:34639 ** 22810 AAAA? clients2.google.com / CNAME: clients.l.google.com, AAAA: 2607:f8b0:4009:80c::200e (ttl 80s) **
dns 2020-03-24 15:57:14      192.168.1.1:53    --      192.168.1.4:38872 ** 56630 A? mtalk.google.com / CNAME: mobile-gtalk.l.google.com, A: 74.125.201.188 (ttl 200s) **
dns 2020-03-24 15:57:14      192.168.1.1:53    --      192.168.1.4:54398 ** 38080 AAAA? clients4.google.com / CNAME: clients.l.google.com, AAAA: 2607:f8b0:4009:813::200e (ttl 65s) **
dns 2020-03-24 15:57:14      192.168.1.1:53    --      192.168.1.4:38872 ** 8511 AAAA? mtalk.google.com / CNAME: mobile-gtalk.l.google.com, AAAA: 2607:f8b0:4001:c14::bc (ttl 169s) **
dns 2020-03-24 15:57:14      192.168.1.1:53    --      192.168.1.4:54398 ** 16312 A? clients4.google.com / CNAME: clients.l.google.com, A: 172.217.8.174 (ttl 144s) **
dns 2020-03-24 15:57:14      192.168.1.1:53    --      192.168.1.4:48611 ** 36628 A? update.googleapis.com / A: 172.217.8.195 (ttl 169s) **
dns 2020-03-24 15:57:14      192.168.1.1:53    --      192.168.1.4:48611 ** 6685 AAAA? update.googleapis.com / AAAA: 2607:f8b0:4009:815::2003 (ttl 249s) **
dns 2020-03-24 15:57:15      192.168.1.1:53    --      192.168.1.4:51568 ** 18214 A? pbs.twimg.com / CNAME: cs196.wac.edgecastcdn.net, CNAME: cs2-wac.apr-8315.edgecastdns.net, CNAME: cs2-wac-us.8315.ecdns.net, CNAME: cs672.wac.edgecastcdn.net, A: 192.229.173.16 (ttl 1607s) **
dns 2020-03-24 15:57:15      192.168.1.1:53    --      192.168.1.4:51568 ** 30000 AAAA? pbs.twimg.com / CNAME: cs196.wac.edgecastcdn.net, CNAME: cs2-wac.apr-8315.edgecastdns.net, CNAME: cs2-wac-us.8315.ecdns.net, CNAME: cs672.wac.edgecastcdn.net, AAAA: 2606:2800:220:13d:2176:94a:948:148e (ttl 282s) **
dns 2020-03-24 15:57:15      192.168.1.1:53    --      192.168.1.4:45840 ** 10482 A? twitter.com / A: 104.244.42.65 (ttl 204s), A: 104.244.42.129 (ttl 204s) **
dns 2020-03-24 15:57:15      192.168.1.1:53    --      192.168.1.4:42808 ** 22387 A? ogs.google.com / CNAME: www3.l.google.com, A: 172.217.9.78 (ttl 79s) **
dns 2020-03-24 15:57:15      192.168.1.1:53    --      192.168.1.4:42808 ** 10107 AAAA? ogs.google.com / CNAME: www3.l.google.com, AAAA: 2607:f8b0:4009:813::200e (ttl 131s) **
dns 2020-03-24 15:57:15      192.168.1.1:53    --      192.168.1.4:53188 ** 47438 AAAA? ssl.gstatic.com / AAAA: 2607:f8b0:4009:803::2003 (ttl 51s) **
dns 2020-03-24 15:57:15      192.168.1.1:53    --      192.168.1.4:53188 ** 46402 A? ssl.gstatic.com / A: 172.217.1.35 (ttl 212s) **
8:58
This is just a simple example, of course. Dshell has some really powerful decoders.

Also realize when decoding from a network interface, your results will not be instantaneous. DShell is both sniffing, and processing the packets. It may take a few seconds before you see output to the screen..

Over time, we get a nice list of DHCP clients..

root@legolas.net.att:/opt/Dshell Dshell> decode -d dhcp -i wlp2s0
WARNING:dhcp:unpack requires a string argument of length 7
dhcp 2020-03-25 08:06:03         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0xcc7bea4a  Client Hostname: DESKTOP-G7F5AG0 Client MAC: 00:db:df:82:39:a8
**
dhcp 2020-03-25 11:45:32         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0x1fd67e34  Client Hostname: android-2dc255debbc724e1 Client MAC: e0:db:10:4f:
58:95   **
dhcp 2020-03-25 11:48:48         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0x3b0ac447  Client Hostname: android-2dc255debbc724e1 Client MAC: c4:1c:ff:a1:
11:62   **
dhcp 2020-03-25 11:48:49         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0x3b0ac447  Client Hostname: android-2dc255debbc724e1 Client MAC: c4:1c:ff:a1:
11:62   **
dhcp 2020-03-25 11:57:47         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0xeef4a9b   Client Hostname: DESKTOP-G7F5AG0 Client MAC: 00:db:df:82:39:a8
**
dhcp 2020-03-25 12:04:07         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0xb096d028  Client Hostname: DESKTOP-G7F5AG0 Client MAC: c4:1c:ff:a1:11:62
**
dhcp 2020-03-25 12:04:08         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0xb096d028  Client Hostname: DESKTOP-G7F5AG0 Client MAC: c4:1c:ff:a1:11:62
**
dhcp 2020-03-25 12:48:32         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0x9224c4a1  Client Hostname: WINDOWS-F8POIBL Client MAC: 00:db:df:f1:0a:1d
**
dhcp 2020-03-25 12:55:12         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0x1968796   Client Hostname: SAMSUNG-SM-G950U Client MAC: dc:ef:ca:ce:fb:eb
 **
dhcp 2020-03-25 14:00:37         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0xffcab1c1  Client Hostname: android-7eb55e3b2dff69c7 Client MAC: c8:a8:23:9a:
bd:36   **
dhcp 2020-03-25 14:23:47         0.0.0.0:68   -- 255.255.255.255:67   ** Transaction ID: 0xa460cc0f  Client Hostname: Living room    Client MAC: c8:a8:23:9a:bd:36

Running Dshell from a pcap

Running dshell against a pcap is as simple as specifying the decoder (-d) then the filename:

decode -d dns test01-200330-00152961.pcap
34.192.146.205 (ttl 43s), A: 3.210.88.27 (ttl 43s), A: 3.214.228.175 (ttl 43s), A: 3.225.196.127 (ttl 43s), A: 3.226.155.212 (ttl 43s), A: 3.230.54.175 (ttl 43s), A: 18.211.68.147 (ttl 43s), A: 18.233.141.189 (ttl 43s) **
dns 2020-03-30 16:15:37   208.67.222.222:53    --      12.68.84.27:48513 ** 43640 A? tps10251.doubleverify.com / CNAME: nycp-hlb.doubleverify.com, CNAME: nycp-hlb.dvgtm.akadns.net, A: 204.154.111.119 (ttl 57s) **
dns 2020-03-30 16:15:37      10.82.17.91:53    --     10.84.81.106:62178 ** 44144 A? sqm.microsoft.com / CNAME: sqmfe.glbdns2.microsoft.com **
dns 2020-03-30 16:15:37      10.82.17.91:53    --    10.84.195.218:52695 ** 40361 A? teams.microsoft.com / CNAME: teams.office.com, CNAME: teams-office-com.s-0005.s-msedge.net, CNAME: s-0005.s-msedge.net, A: 52.113.194.132 (ttl 74s) **
dns 2020-03-30 16:15:37      10.70.12.72:53    --     10.84.196.47:60102 ** 38924 A? eastus.notifications.teams.microsoft.com / CNAME: eastuscns.trafficmanager.net, CNAME: eastuscns-16.eastus.cloudapp.azure.com, A: 52.114.133.8 (ttl 10s) **
A: 67.226.212.59 (ttl 163s), A: 67.226.212.60 (ttl 163s), A: 67.226.212.55 (ttl 163s), A: 67.226.212.56 (ttl 163s) **
dns 2020-03-30 16:15:37      10.82.17.91:53    --    10.84.194.198:56018 ** 33928 A? southcentralus.notifications.teams.microsoft.com / CNAME: southcentraluscns.trafficmanager.net, CNAME: southcentraluscns-28.southcentralus.cloudapp.azure.com, A: 52.114.144.25 (ttl 10s) **
dns 2020-03-30 16:15:37      10.70.12.72:53    --    10.84.194.198:56018 ** 33928 A? southcentralus.notifications.teams.microsoft.com / CNAME: southcentraluscns.trafficmanager.net, CNAME: southcentraluscns-28.southcentralus.cloudapp.azure.com, A: 52.114.144.25 (ttl 4s) **
dns 2020-03-30 16:15:37      10.82.17.91:53    --     10.84.196.72:55455 ** 29864 A? cdn.syndication.twimg.com / CNAME: cs196.wac.edgecastcdn.net, CNAME: cs2-wac.apr-8315.edgecastdns.net, CNAME: cs2-wac-us.8315.ecdns.net, CNAME: cs45.wac.edgecastcdn.net, A: 72.21.91.70 (ttl 882s) **
dns 2020-03-30 16:15:37      10.82.17.91:53    --     10.84.81.106:51938 ** 44545 A? sqm.microsoft.com / CNAME: sqmfe.glbdns2.microsoft.com **
dns 2020-03-30 16:15:37   208.67.220.220:53    --      12.68.84.27:54400 ** 45921 A? acp-ss.adobe.io / A: 52.0.111.85 (ttl 60s), A: 52.0.166.103 (ttl 60s), A: 52.1.255.31 (ttl 60s), A: 52.20.40.93 (ttl 60s), A: 52.21.18.122 (ttl 60s), A: 52.22.128.58 (ttl 60s), A: 35.172.75.11 (ttl 60s), A: 52.0.102.100 (ttl 60s) **

Blog Archive