Ethical Hacking: Enumerating and Fingerprinting Systems and Applications

Details about the HTTP Protocol

Web applications use many different protocols, the most prevalent of which is HTTP. This book assumes that you have a basic understanding of Internet protocols and their use, but this chapter takes a deep dive into the components of protocols like HTTP that you will find in nearly all web applications.

The HTTP Protocol

Let’s look at a few facts and definitions before we proceed to details about HTTP:

  • The HTTP 1.1 protocol is defined in RFC 7230RFC 7231RFC 7232RFC 7233RFC 7234, and RFC 7235.
  • HTTP/2 (HTTP version 2.0) is defined in RFC 7540.
  • In the examples in this lab, when we refer to an HTTP server, we basically mean a web server.
  • When we refer to HTTP clients, we are talking about browsers, proxies, API clients, and other custom HTTP client programs.
  • HTTP is a very simple protocol, which is both a good thing and a bad thing.
  • In most cases, HTTP is categorized as a stateless protocol that does not rely on a persistent connection for communication logic.
  • An HTTP transaction consists of a single request from a client to a server, followed by a single response from the server back to the client.
  • HTTP is different from stateful protocols, such as FTP, SMTP, IMAP, and POP. When a protocol is stateful, sequences of related commands are treated as a single interaction.
  • A server must maintain the state of its interaction with the client throughout the transmission of successive commands, until the interaction is terminated.
  • A sequence of transmitted and executed commands is often called a session.

What about HTTPS?

Hypertext Transfer Protocol Secure (HTTPS) is an extension of HTTP. HTTPS is the preferred method of communication for web applications and application programming interfaces (APIs), since the communication protocol is encrypted using Transport Layer Security (TLS) or, formerly, Secure Sockets Layer (SSL). The protocol is therefore also referred to as HTTP over TLS, or HTTP over SSL.

HTTP Proxies

HTTP proxies act as both servers and clients. Proxies make requests to web servers on behalf of other clients. They enable HTTP transfers across firewalls and can also provide support for caching of HTTP messages. Proxies can perform other roles in complex environments, including Network Address Translation (NAT) and filtering of HTTP requests.

There is another type of a “web proxy”. There are tools such as Burp Suite and the OWASP Zed Attack Proxy (ZAP) that allow you to intercept communications between a web browser or a client and a web server. Burp Suite and the OWASP ZAP are the most popular tools among pen testers, security researchers, and bug hunters that offer proxy capabilities.

Request and Response Model

HTTP is an application-level protocol in the TCP/IP protocol suite, and it uses TCP as the underlying transport layer protocol for transmitting messages. HTTP uses a request/response model, which basically means that an HTTP client program sends an HTTP request message to a server, and then the server returns an HTTP response message.

HTTP Methods

As an ethical hacker, you must be familiar with the different HTTP methods. The following are the different HTTP methods:

  • GET: Retrieves information from the server
  • HEAD: Basically the same as GET but returns only HTTP headers and no document body
  • POST: Sends data to the server (typically using HTML forms, API requests, and so on)
  • TRACE: Does a message loopback test along the path to the target resource
  • PUT: Uploads a representation of the specified URI
  • DELETE: Deletes the specified resource
  • OPTIONS: Returns the HTTP methods that the server supports
  • CONNECT: Converts the request connection to a transparent TCP/IP tunnel

Let’s use the curl tool to display the header information of an HTTP GET request to a web application running on a system with the IP address, as well as the response coming back from the web server.

curl -Ivs

You should see the details about the GET Request and the response back from the web server/web application, similar to the output below:

$ curl -Ivs
*   Trying
* Connected to ( port 80 (#0)
> HEAD / HTTP/1.1
> Host:
> User-Agent: curl/7.68.0
> Accept: */*
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: <today's date and time>
Date: <today's date and time>
< Server: Apache/2.4.10 (Debian)
Server: Apache/2.4.10 (Debian)
< Content-Type: text/html; charset=UTF-8
Content-Type: text/html; charset=UTF-8

Status Codes

The web server response includes a three-digit status code and a brief human-readable explanation of the status code. Below that you can see the text data (which is the HTML code coming back from the server and displaying the website contents).

Tip: It is important that you become familiar with HTTP message status codes. W3Schools provides a very good explanation at their website

The HTTP status code messages can be in the following ranges:

  • Messages in the 100 range are informational.
  • Messages in the 200 range are related to successful transactions.
  • Messages in the 300 range are related to HTTP redirections.
  • Messages in the 400 range are related to client errors.
  • Messages in the 500 range are related to server errors.

Web Sessions

A web session is a sequence of HTTP request and response transactions between a web client and a server. These transactions include pre-authentication tasks, the authentication process, session management, access control, and session finalization. Numerous web applications keep track of information about each user for the duration of a web transactions. Several web applications have the ability to establish variables such as access rights and localization settings. These variables apply to each and every interaction a user has with the web application for the duration of the session.

Web applications can create sessions to keep track of anonymous users after the very first user request. For example, an application can remember the user language preference every time it visits the site or application front end. In addition, a web application uses a session after the user has authenticated. This allows the application to identify the user on any subsequent requests and also to be able to apply security access controls and to increase the usability of the application. In short, web applications can provide session capabilities both before and after authentication.

After an authenticated session has been established, the session ID (or token) is temporarily equivalent to the strongest authentication method used by the application, such as usernames and passwords, one-time passwords, and client-based digital certificates.

TIP: A good resource that provides a lot of information about application authentication is [the OWASP Authentication Cheat Sheet](].

In order to keep the authenticated state and track user progress, applications provide users with session IDs, or tokens. A token is assigned at session creation time, and it is shared and exchanged by the user and the web application for the duration of the session. The session ID is a name/value pair.

The session ID names used by the most common web application development frameworks can be easily fingerprinted. For instance, you can easily fingerprint PHPSESSID (PHP), JSESSIONID (J2EE), CFID and CFTOKEN (ColdFusion), ASP.NET_SessionId (ASP .NET), and many others. In addition, the session ID name may indicate what framework and programming languages are used by the web application.

It is recommended to change the default session ID name of the web development framework to a generic name, such as id.

The session ID must be long enough to prevent brute-force attacks. Sometimes developers set it to just a few bits, though it must be at least 128 bits (16 bytes).

It is very important that a session ID be unique and unpredictable. You should use a good deterministic random bit generator (DRBG) to create a session ID value that provides at least 256 bits of entropy.

There are multiple mechanisms available in HTTP to maintain session state within web applications, including cookies (in the standard HTTP header), the URL parameters and rewriting defined in RFC 3986, and URL arguments on GET requests. In addition, developers use body arguments on POST requests, such as hidden form fields (HTML forms) or proprietary HTTP headers. However, one of the most widely used session ID exchange mechanisms is cookies, which offer advanced capabilities not available in other methods. Including the session ID in the URL can lead to the manipulation of the ID or session fixation attacks. It is therefore important to keep the session ID out of the URL.

Web development frameworks such as ASP .NET, PHP, and Ruby on Rails provide their own session management features and associated implementations. It is recommended to use these built-in frameworks rather than build your own from scratch, since they have been tested by many people. When you perform pen testing, you are likely to find people trying to create their own frameworks. In addition, JSON Web Token (JWT) can be used for authentication in modern applications.

This may seem pretty obvious, but you have to remember to encrypt an entire web session, not only for the authentication process where the user credentials are exchanged but also to ensure that the session ID is exchanged only through an encrypted channel. The use of an encrypted communication channel also protects the session against some session fixation attacks, in which the attacker is able to intercept and manipulate the web traffic to inject (or fix) the session ID on the victim’s web browser.

Session management mechanisms based on cookies can make use of two types of cookies: non-persistent (or session) cookies and persistent cookies. If a cookie has a Max-Age or Expires attribute, it is considered a persistent cookie and is stored on a disk by the web browser until the expiration time. Common web applications and clients prioritize the Max-Age attribute over the Expires attribute.

Modern applications typically track users after authentication by using non-persistent cookies. This forces the session information to be deleted from the client if the current web browser instance is closed. This is why it is important to use nonpersistent cookies: so the session ID does not remain on the web client cache for long periods of time.

Session IDs must be carefully validated and verified by an application. Depending on the session management mechanism that is used, the session ID will be received in a GET or POST parameter, in the URL, or in an HTTP header using cookies.

If web applications do not validate and filter out invalid session ID values, they can potentially be used to exploit other web vulnerabilities, such as SQL injection if the session IDs are stored on a relational database or persistent cross-site scripting (XSS) if the session IDs are stored and reflected back afterward by the web application.

ntroducing Banner Grabbing and Enumeration

Banner grabbing is referred to the act when you gain information about a system and services running on its open ports. So, what’s a banner? A banner is a text displayed by the targeted system that contains information about the software type, technology, and version running in the system.

Performing Banner Grabbing

In the Introducing Network and Port Scanning using Nmap and the Advanced Nmap Scanning and the Nmap Scripting Engine scenarios, you learned that you can use Nmap to perform active reconnaissance by scanning systems and getting software banner information, such as name and version.

As a refresher, install Nmap and perform a network discovery scan on the network to find the two target systems available in this lab:

sudo apt update; sudo apt install nmap -y

You can perform a network discovery scan by using the -sn option, as demonstrated below:

nmap -sn

You should reveal the systems with the IP address and

You can perform an aggresive scan to find information about the two systems with the following commands:

nmap -A


nmap -A

  • What ports were you able to find open in bboth systems?
  • Can you determine the web server name and type of both systems?

HINT: A Nmap cheat sheet provided by can be viewed by visiting, or simply using curl and accessing

Guess what? You can even use the Linux curl or wget utilities to perform basic banner grabbing of web applications, as demonstrated below:

curl -Ivs

You should be able to obtain information about the underlying web server software (Apache version 2.4.10).

Let’s now use wget to obtain the web server type and version of the system running on

wget -q -S

The system is running nginx version 1.14.2.

You can even use the telnet command to perform banner grabbing, as demonstrated below:

telnet 80

Enumeration and Banner Grabbing with WhatWeb

Whatweb is a tool that can be used to disclose the web server information such as the IP address, version, webpage title, and running operating system.

You can install whatweb with the following command:

sudo apt update && apt install whatweb -y

Whatweb supports numerous options to perform reconnaissance. Access the help and command syntax by executing the following command:

whatweb -h

Notice that you can specify different aggression level controls (the default is Stealthy):

The aggression level controls the trade-off between speed/stealth and
  --aggression, -a=LEVEL        Set the aggression level. Default: 1.
  1. Stealthy                   Makes one HTTP request per target and also
                                follows redirects.
  3. Aggressive                 If a level 1 plugin is matched, additional
                                requests will be made.
  4. Heavy                      Makes a lot of HTTP requests per target. URLs
                                from all plugins are attempted.

Let’s perform an “aggresive” enumeration of the web application running on

whatweb --aggression 3

Recon with DMitry

The Deepmagic Information Gathering Tool (DMitry) can be used to perform enumeration, banner grabbing, and other recon activities. You can install with the following command:

sudo apt update && apt install dmitry -y

Once installed, access the command syntax help using:

dmitry -h

Let’s perform basic enumeration and banner grabbing of the web application and system running on

dmitry -b

  • Where you able to find the FTP server type and version?
  • What is the OpenSSH version used by the targeted system?

ntroduction to Vulnerability Scanning

The following interactive scenarios include information about different scanning technologies and solutions such as Nmap and the web application scanner called Nikto:

Nmap provides a user with a powerful toolset of features, that allows the user to create scripts to complete custom tasks. Scripts are created using the LUA scripting language and end with the .nse file extension. There are well over 300+ examples to choose from, and many are installed with Nmap. The Nmap documentation provides a complete comprehensive list of scripts that ship with Nmap when installing.

OWASP Zed Attack Proxy

You can use the OWASP Zed Attack Proxy (ZAP) to perform comprehensive vulnerability scanning of web applications. The OWASP ZAP full scan is a script that is available in the ZAP Docker images.

You can obtain the OWASP ZAP Docker container stable release by executing the following command:

docker pull owasp/zap2docker-stable

In this lab, we will use the stable release. However, for your reference only, you can obtain the the latest weekly release by executing the following command:

docker pull owasp/zap2docker-weekly

You can obtain the the live release (built whenever the zaproxy project is changed):

docker pull owasp/zap2docker-live

You can obtain the the bare release using the following command:

docker pull owasp/zap2docker-bare

NOTE: The bare release is a very small Docker image that contains only the necessary required dependencies to run OWASP ZAP. This makes it ideal for continuous integration (CI) environments.

The Dockerfiles can be found from the OWASP ZAP GitHub repository.

In the following step, you will learn how to launch a full scan using the OWASP ZAP.

Web Server Vulnerability Identification

The OWASP ZAP runs the ZAP spider against the specified target followed by an optional ajax spider scan and then a full active scan before reporting the results. In other words, the script performs actual ‘attacks’ and can potentially run for a long period of time.

Execute the following command to perform a full scan:

docker run -t owasp/zap2docker-stable -t

By default, the OWASP ZAP reports all alerts as WARNings but you can specify a config file which can change any rules to FAIL or IGNORE. The configuration works in a very similar way as the Baseline Scan so see the Baseline page for more details.

NetBIOS Enumeration

On the Introducing Network and Port Scanning using Nmap and the Advanced Nmap Scanning and the Nmap Scripting Engine scenarios you learned details about the Nmap scanner.

The following is a good trick to enumerate any devices that are running NetBIOS in a given subnet.

First, install Nmap: apt update && apt install nmap -y

Then perform the following scan to find any systems that are running the default NetBIOS ports (139 and 445):

nmap -v -p 139,445 -oG netbios.txt

The output of the scan is also saved in a file called netbios.txt. Open the file to analyze the results of the scan: cat netbios.txt

Additional Enumeration using the Nmap Scripting Engine

Now that you know that the system with IP address is running NetBIOS, you can perform additional enumeration using the following command:

nmap -p 139,445 -sC

The -sC options launches the most common Nmap Scritpting Engine (NSE) scripts against the services running (i.e., open ports) in the target system. You can search for all the smb related NSE scripts by executing the following command:

ls /usr/share/nmap/scripts/ | grep smb

Let’s try to enumerate the system’s shares using the following command:

nmap --script smb-enum-shares.nse

Using Metasploit

Metasploit is a very popular exploitation framework used by bug hunters and penetration testers. It includes several auxiliary modules that can be used to enumerate systems and applications.

We will not install Metasploit in this lab. However, the following are the Metasploit commands for performing SMB enumeration using the Metasploit auxiliary module.

Enumerating the SMB version

The following is an example on how to enumerate the SMB version used in a Samba installation using Metasploit.

msf > use auxiliary/scanner/smb/smb_version
msf auxiliary(scanner/smb/smb_version) > set RHOSTS
msf auxiliary(scanner/smb/smb_version) > run
[*]    - Host could not be identified: Unix (Samba 2.2.1a)

SMB Brute Force

The following Metasploit auxiliary module can be used to perform an SMB Brute Force attack:

msf> use auxiliary/scanner/smb/smb_login

Existing users

msf auxiliary(scanner/smb/smb_lookupsid) > use auxiliary/scanner/smb/smb_lookupsid
msf auxiliary(scanner/smb/smb_lookupsid) > set RHOSTS
msf auxiliary(scanner/smb/smb_lookupsid) > run
[*]    - PIPE(LSARPC) LOCAL(MYGROUP - 5-21-4157223341-3243572438-1405127623) DOMAIN(MYGROUP - )
[*]    - TYPE=0 NAME=Administrator rid=500

Enumerating Samba Implementations
Samba enables Linux hosts to communicate with systems running Windows in a network. Samba is open source software originally developed for file and print share for all clients using the SMB protocol. However, it has evolved and added more capabilities. Attackers have abused many Samba and SMB implementations to perform different attacks. The following are some additional tips on how to enumerate Samba implementations.

Accessing the Attacking System
In this step, we will use an attacking system running the Parrot Security OS in a Docker container. Type the following command to access the container:

docker exec -it attacker bash

Your prompt should change to the following:

└──╼ #
Note: The container should've been launched automatically when you started this lab. If you have trouble accessing the container, verify that is running by using the docker ps command. Alternatively, you can manually launch all the containers needed by this lab by executing bash

Enumerating Samba Implementations using enum4Linux and enum4linux-ng
In the previous step you learned how to use Nmap and Metasploit to perform SMB enumeration. There are other tools that can be used to perform further enumeration. One of the most popular tools is enum4linux and a new alternative called enum4linux-ng.

In this lab we will use enum4linux-ng. Complete the following steps to install and run enum4linux-ng:

Step 1. From the attacker system, clone the GitHub repository:

git clone

Step 2. enum4linux-ng was created using Python. Install the Python 3 pip to install the required dependencies:

apt update && apt install -y python3-pip

Step 3. Access the enum4linux-ng directory:

cd enum4linux-ng

Step 4. Install the required dependencies using pip3: pip3 install -r requirements.txt

Step 5. You should now be able to run enum4linux-ng and access the help menu: #python3  -h

The output should be similar to the text below:

ENUM4LINUX - next generation

usage: [-h] [-A] [-As] [-U] [-G] [-Gm] [-S] [-C] [-P] [-O] [-L] [-I] [-R] [-N] [-w WORKGROUP] [-u USER] [-p PW] [-d] [-k USERS] [-r RANGES] [-s SHARES_FILE]
                        [-t TIMEOUT] [-v] [--keep] [-oJ OUT_JSON_FILE | -oY OUT_YAML_FILE | -oA OUT_FILE]

This tool is a rewrite of Mark Lowe's, a tool for enumerating information from Windows and Samba systems. It is mainly a wrapper around the Samba tools nmblookup, net,
rpcclient and smbclient. Other than the original tool it allows to export enumeration results as YAML or JSON file, so that it can be further processed with other tools. The tool
tries to do a 'smart' enumeration. It first checks whether SMB or LDAP is accessible on the target. Depending on the result of this check, it will dynamically skip checks (e.g. LDAP
checks if LDAP is not running). If SMB is accessible, it will always check whether a session can be set up or not. If no session can be set up, the tool will stop enumeration. The
enumeration process can be interupted with CTRL+C. If the options -oJ or -oY are provided, the tool will write out the current enumeration state to the JSON or YAML file, once it
receives SIGINT triggered by CTRL+C. The tool was made for security professionals and CTF players. Illegal use is prohibited.

positional arguments:

optional arguments:
  -h, --help         show this help message and exit
  -A                 Do all simple enumeration including nmblookup (-U -G -S -P -O -N -I -L). This option is enabled if you don't provide any other option.
  -As                Do all simple short enumeration without NetBIOS names lookup (-U -G -S -P -O -I -L)
  -U                 Get users via RPC
  -G                 Get groups via RPC
  -Gm                Get groups with group members via RPC
  -S                 Get shares via RPC
  -C                 Get services via RPC
  -P                 Get password policy information via RPC
  -O                 Get OS information via RPC
  -L                 Get additional domain info via LDAP/LDAPS (for DCs only)
  -I                 Get printer information via RPC
  -R                 Enumerate users via RID cycling
  -N                 Do an NetBIOS names lookup (similar to nbtstat) and try to retrieve workgroup from output
  -w WORKGROUP       Specify workgroup/domain manually (usually found automatically)
  -u USER            Specify username to use (default "")
  -p PW              Specify password to use (default "")
  -d                 Get detailed information for users and groups, applies to -U, -G and -R
  -k USERS           User(s) that exists on remote system (default: administrator,guest,krbtgt,domain admins,root,bin,none). Used to get sid with "lookupsid known_username"
  -r RANGES          RID ranges to enumerate (default: 500-550,1000-1050)
  -s SHARES_FILE     Brute force guessing for shares
  -t TIMEOUT         Sets connection timeout in seconds (default: 5s)
  -v                 Verbose, show full samba tools commands being run (net, rpcclient, etc.)
  --keep             Don't delete the Samba configuration file created during tool run after enumeration (useful with -v)
  -oJ OUT_JSON_FILE  Writes output to JSON file (extension is added automatically)
  -oY OUT_YAML_FILE  Writes output to YAML file (extension is added automatically)
  -oA OUT_FILE       Writes output to YAML and JSON file (extensions are added automatically)
Step 6. Run the following command to enumerate information about the intentionally vulnerable system ( python3 -As

The output should be similar to the text below. Notice that the tool was able to enumerate two users, SMB shares, and other information of the target system.

ENUM4LINUX - next generation

|    Target Information    |
[*] Target ...........
[*] Username ......... ''
[*] Random Username .. 'nmknyyro'
[*] Password ......... ''
[*] Timeout .......... 5 second(s)

|    Service Scan on    |
[*] Checking LDAP
[-] Could not connect to LDAP on 389/tcp: connection refused
[*] Checking LDAPS
[-] Could not connect to LDAPS on 636/tcp: connection refused
[*] Checking SMB
[+] SMB is accessible on 445/tcp
[*] Checking SMB over NetBIOS
[+] SMB over NetBIOS is accessible on 139/tcp

|    SMB Dialect Check on    |
[*] Trying on 445/tcp
[+] Supported dialects and settings:
SMB 1.0: true
SMB 2.02: true
SMB 2.1: true
SMB 3.0: true
SMB1 only: false
Preferred dialect: SMB 3.0
SMB signing required: false

|    RPC Session Check on    |
[*] Check for null session
[+] Server allows session using username '', password ''
[*] Check for random user session
[-] Could not establish random user session: STATUS_LOGON_FAILURE

|    Domain Information via RPC for    |
[+] Domain: WORKGROUP
[+] Host is part of a workgroup (not a domain)

|    Domain Information via SMB session for    |
[*] Enumerating via unauthenticated SMB session on 445/tcp
[+] Found domain information via SMB
NetBIOS computer name: GRAVEMIND
NetBIOS domain name: ''
DNS domain: ''
FQDN: gravemind

|    OS Information via RPC for    |
[*] Enumerating via unauthenticated SMB session on 445/tcp
[+] Found OS information via SMB
[*] Enumerating via 'srvinfo'
[+] Found OS information via 'srvinfo'
[+] After merging OS information we have the following result:
OS: Linux/Unix (Samba 4.9.5-Debian)
OS version: '6.1'
OS release: ''
OS build: '0'
Native OS: Windows 6.1
Native LAN manager: Samba 4.9.5-Debian
Platform id: '500'
Server type: '0x809a03'
Server type string: Wk Sv PrQ Unx NT SNT Samba 4.9.5-Debian

|    Users via RPC on    |
[*] Enumerating users via 'querydispinfo'
[+] Found 2 users via 'querydispinfo'
[*] Enumerating users via 'enumdomusers'
[+] Found 2 users via 'enumdomusers'
[+] After merging user results we have 2 users total:
  username: masterchief
  name: ''
  acb: '0x00000015'
  description: ''
  username: arbiter
  name: ''
  acb: '0x00000015'
  description: ''

|    Groups via RPC on    |
[*] Enumerating local groups
[+] Found 0 group(s) via 'enumalsgroups domain'
[*] Enumerating builtin groups
[+] Found 0 group(s) via 'enumalsgroups builtin'
[*] Enumerating domain groups
[+] Found 0 group(s) via 'enumdomgroups'

|    Shares via RPC on    |
[*] Enumerating shares
[+] Found 4 share(s):
  comment: IPC Service (Samba 4.9.5-Debian)
  type: IPC
  comment: All home directories
  type: Disk
  comment: Printer Drivers
  type: Disk
  comment: Confidential Workfiles
  type: Disk
[*] Testing share IPC$
[-] Could not check share: STATUS_OBJECT_NAME_NOT_FOUND
[*] Testing share homes
[-] Share doesn't exist
[*] Testing share print$
[+] Mapping: OK, Listing: OK
[*] Testing share workfiles
[+] Mapping: OK, Listing: OK

|    Policies via RPC for    |
[*] Trying port 445/tcp
[+] Found policy:
  pw_history_length: None
  min_pw_length: 5
  min_pw_age: none
  max_pw_age: 49710 days 6 hours 21 minutes
  lockout_observation_window: 30 minutes
  lockout_duration: 30 minutes
  lockout_threshold: None
  force_logoff_time: 49710 days 6 hours 21 minutes

|    Printers via RPC for    |
[+] No printers returned (this is not an error)

Completed after 12.35 seconds
TIP: This chapter provides additional details about how to perform information gathering and vulnerability identification