How To: Fingerprint Web Apps & Servers for Better Recon & More Successful Hacks

Web applications are ubiquitous in the modern online world, and knowing how to attack them is an increasingly valuable skill. But the key to a successful attack is good recon since it's easier to be focused and efficient with the more information you have. There are many fingerprinting tools available, such as httprint and WebTech, but there are even more that can aid us in reconnaissance.

Common Frameworks & Technologies

Gone are the days of simple websites using HTML, CSS, and vanilla JavaScript. Frameworks dominate the landscape today, providing a robust and modular approach to modern web development. And with more complicated web apps comes more data, so there are now more types of databases than ever. MySQL, SQL Server, and Oracle are still around, but newer players like Redis, PostgreSQL, and MongoDB are gaining popularity.

• Don't Miss: How to Fingerprint an Entire Network Using a Domain Name

As far as frameworks go, JavaScript-based ones are arguably the most popular. React, Angular, and Node continue to be widely used, as well as Meteor, Ember, and Backbone. In the PHP arena, you have Symfony, Cake, and Laravel. Django and Flask are both tried-and-true frameworks built on Python, and of course, there is Microsoft's ASP.NET and Ruby on Rails.

Method 1: Netcat

Netcat is a popular networking tool used to troubleshoot and communicate via TCP/IP. For most hackers, what immediately comes to mind is using it for reverse shells, but it can also be used to fingerprint web servers. To initiate a connection, simply supply the host and port you wish to connect to:

~# nc google.com 80

█

GET / HTTP/1.1
Host: google.com

Now, it will seem like nothing happened; we have to issue a command, in this case, a GET request. Press Enter and specify the host.

GET / HTTP/1.1
Host: google.com

Press Enter twice again and we should see the response:

HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Mon, 04 May 2021 15:57:50 GMT
Expires: Wed, 03 Jun 2021 15:57:50 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN

...

We can see that this is a 301 redirect (since Google uses HTTPS), but we can also see the type of server and some header information.

Method 2: WhatWeb

The next tool we'll use to fingerprint is WhatWeb, a scanner specifically designed to gather information about a web application or server. Type whatweb at the terminal to bring up the help:

~# whatweb

.$$$     $.                                   .$$$     $.
$$$$     $$. .$$$  $$$ .$$$$$$.  .$$$$$$$$$$. $$$$     $$. .$$$$$$$. .$$$$$$.
$ $$     $$$ $ $$  $$$ $ $$$$$$. $$$$$ $$$$$$ $ $$     $$$ $ $$   $$ $ $$$$$$.
$ `$     $$$ $ `$  $$$ $ `$  $$$ $$' $ `$ `$$ $ `$     $$$ $ `$      $ `$  $$$'
$. $     $$$ $. $$$$$$ $. $$$$$$ `$  $. $  :' $. $     $$$ $. $$$$   $. $$$$$.
$::$  .  $$$ $::$  $$$ $::$  $$$     $::$     $::$  .  $$$ $::$      $::$  $$$$
$;;$ $$$ $$$ $;;$  $$$ $;;$  $$$     $;;$     $;;$ $$$ $$$ $;;$      $;;$  $$$$
$$$$$$ $$$$$ $$$$  $$$ $$$$  $$$     $$$$     $$$$$$ $$$$$ $$$$$$$$$ $$$$$$$$$'

WhatWeb - Next generation web scanner version 0.5.1.
Developed by Andrew Horton (urbanadventurer) and Brendan Coles (bcoles)
Homepage: https://www.morningstarsecurity.com/research/whatweb

Usage: whatweb [options] <URLs>

  <TARGETs>                     Enter URLs, hostnames, IP addresses, filenames or
                                IP ranges in CIDR, x.x.x-x, or x.x.x.x-x.x.x.x
                                format.
  --input-file=FILE, -i         Read targets from a file.

  --aggression, -a=LEVEL        Set the aggression level. Default: 1.
  1. Stealthy                   Makes one HTTP request per target and also
                                follows redirects.
  3. Aggressive                 If a level 1 plugin is matched, additional
                                requests will be made.

  --list-plugins, -l            List all plugins.
  --info-plugins, -I=[SEARCH]   List all plugins with detailed information.
                                Optionally search with a keyword.

  --verbose, -v                 Verbose output includes plugin descriptions.

Note: This is the short usage help. For the complete usage help use -h or --help.

This gives us basic usage and a few options, enough to use the tool successfully. More detailed help information can be viewed using the -h flag:

~# whatweb -h

.$$$     $.                                   .$$$     $.
$$$$     $$. .$$$  $$$ .$$$$$$.  .$$$$$$$$$$. $$$$     $$. .$$$$$$$. .$$$$$$.
$ $$     $$$ $ $$  $$$ $ $$$$$$. $$$$$ $$$$$$ $ $$     $$$ $ $$   $$ $ $$$$$$.
$ `$     $$$ $ `$  $$$ $ `$  $$$ $$' $ `$ `$$ $ `$     $$$ $ `$      $ `$  $$$'
$. $     $$$ $. $$$$$$ $. $$$$$$ `$  $. $  :' $. $     $$$ $. $$$$   $. $$$$$.
$::$  .  $$$ $::$  $$$ $::$  $$$     $::$     $::$  .  $$$ $::$      $::$  $$$$
$;;$ $$$ $$$ $;;$  $$$ $;;$  $$$     $;;$     $;;$ $$$ $$$ $;;$      $;;$  $$$$
$$$$$$ $$$$$ $$$$  $$$ $$$$  $$$     $$$$     $$$$$$ $$$$$ $$$$$$$$$ $$$$$$$$$'

WhatWeb - Next generation web scanner version 0.5.1.
Developed by Andrew Horton (urbanadventurer) and Brendan Coles (bcoles).
Homepage: https://www.morningstarsecurity.com/research/whatweb

Usage: whatweb [options] <URLs>

TARGET SELECTION:
  <TARGETs>                     Enter URLs, hostnames, IP addresses, filenames or
                                IP ranges in CIDR, x.x.x-x, or x.x.x.x-x.x.x.x
                                format.
  --input-file=FILE, -i         Read targets from a file. You can pipe
                                hostnames or URLs directly with -i /dev/stdin.

TARGET MODIFICATION:
  --url-prefix                  Add a prefix to target URLs.
  --url-suffix                  Add a suffix to target URLs.
  --url-pattern                 Insert the targets into a URL.
                                e.g. example.com/%insert%/robots.txt

AGGRESSION:
The aggression level controls the trade-off between speed/stealth and
reliability.
  --aggression, -a=LEVEL        Set the aggression level. Default: 1.
  1. Stealthy                   Makes one HTTP request per target and also
                                follows redirects.
  3. Aggressive                 If a level 1 plugin is matched, additional
                                requests will be made.
  4. Heavy                      Makes a lot of HTTP requests per target. URLs
                                from all plugins are attempted.

HTTP OPTIONS:
  --user-agent, -U=AGENT        Identify as AGENT instead of WhatWeb/0.5.1.
  --header, -H                  Add an HTTP header. eg "Foo:Bar". Specifying a
                                default header will replace it. Specifying an
                                empty value, e.g. "User-Agent:" will remove it.
  --follow-redirect=WHEN        Control when to follow redirects. WHEN may be
                                `never', `http-only', `meta-only', `same-site',
                                or `always'. Default: always.
  --max-redirects=NUM           Maximum number of redirects. Default: 10.

...

EXAMPLE USAGE:
* Scan example.com.
  ./whatweb example.com

* Scan reddit.com slashdot.org with verbose plugin descriptions.
  ./whatweb -v reddit.com slashdot.org

* An aggressive scan of wired.com detects the exact version of WordPress.
  ./whatweb -a 3 www.wired.com

* Scan the local network quickly and suppress errors.
  whatweb --no-errors 192.168.0.0/24

* Scan the local network for https websites.
  whatweb --no-errors --url-prefix https:// 192.168.0.0/24

* Scan for crossdomain policies in the Alexa Top 1000.
  ./whatweb -i plugin-development/alexa-top-100.txt \
  --url-suffix /crossdomain.xml -p crossdomain_xml

We can see more detailed options now and a few more usage examples. The simplest way to run WhatWeb is by giving it a host to scan:

~# whatweb google.com

/usr/lib/ruby/vendor_ruby/target.rb:188: warning: URI.escape is obsolete
/usr/lib/ruby/vendor_ruby/target.rb:188: warning: URI.escape is obsolete
http://google.com [301 Moved Permanently] Country[UNITED STATES][US], HTTPServer[gws], IP[172.217.6.14], RedirectLocation[http://www.google.com/], Title[301 Moved], X-Frame-Options[SAMEORIGIN], X-XSS-Protection[0]
http://www.google.com/ [200 OK] Cookies[1P_JAR,NID], Country[UNITED STATES][US], HTML5, HTTPServer[gws], HttpOnly[NID], IP[172.217.4.36], Script, Title[Google], X-Frame-Options[SAMEORIGIN], X-XSS-Protection[0]

It throws a couple of errors (which can be ignored) and spits out some information about the web server, including HTTP status codes, cookie information, and an IP address. This is certainly useful, but the output is a little hard to read — we can use the -v option to give us verbose output in a format that's much easier on the eyes:

~# whatweb google.com -v

/usr/lib/ruby/vendor_ruby/target.rb:188: warning: URI.escape is obsolete
/usr/lib/ruby/vendor_ruby/target.rb:188: warning: URI.escape is obsolete
WhatWeb report for http://google.com
Status    : 301 Moved Permanently
Title     : 301 Moved
IP        : 172.217.6.14
Country   : UNITED STATES, US

Summary   : X-Frame-Options[SAMEORIGIN], X-XSS-Protection[0], RedirectLocation[http://www.google.com/], HTTPServer[gws]

Detected Plugins:
[ HTTPServer ]
        HTTP server header string. This plugin also attempts to
        identify the operating system from the server header.

        String       : gws (from server string)

[ RedirectLocation ]
        HTTP Server string location. used with http-status 301 and
        302

        String       : http://www.google.com/ (from location)

[ X-Frame-Options ]
        This plugin retrieves the X-Frame-Options value from the
        HTTP header. - More Info:
        http://msdn.microsoft.com/en-us/library/cc288472%28VS.85%29.
        aspx

        String       : SAMEORIGIN

[ X-XSS-Protection ]
        This plugin retrieves the X-XSS-Protection value from the
        HTTP header. - More Info:
        http://msdn.microsoft.com/en-us/library/cc288472%28VS.85%29.
        aspx

        String       : 0

HTTP Headers:
        HTTP/1.1 301 Moved Permanently
        Location: http://www.google.com/
        Content-Type: text/html; charset=UTF-8
        Date: Mon, 04 May 2021 16:09:45 GMT
        Expires: Wed, 03 Jun 2021 16:09:45 GMT
        Cache-Control: public, max-age=2592000
        Server: gws
        Content-Length: 219
        X-XSS-Protection: 0
        X-Frame-Options: SAMEORIGIN
        Connection: close

WhatWeb report for http://www.google.com/
Status    : 200 OK
Title     : Google
IP        : 172.217.4.36
Country   : UNITED STATES, US

Summary   : X-Frame-Options[SAMEORIGIN], X-XSS-Protection[0], Cookies[1P_JAR,NID], HttpOnly[NID], Script, HTML5, HTTPServer[gws]

Detected Plugins:
[ Cookies ]
        Display the names of cookies in the HTTP headers. The
        values are not returned to save on space.

        String       : 1P_JAR
        String       : NID

[ HTML5 ]
        HTML version 5, detected by the doctype declaration

[ HTTPServer ]
        HTTP server header string. This plugin also attempts to
        identify the operating system from the server header.

        String       : gws (from server string)

[ HttpOnly ]
        If the HttpOnly flag is included in the HTTP set-cookie
        response header and the browser supports it then the cookie
        cannot be accessed through client side script - More Info:
        http://en.wikipedia.org/wiki/HTTP_cookie

        String       : NID

[ Script ]
        This plugin detects instances of script HTML elements and
        returns the script language/type.

[ X-Frame-Options ]
        This plugin retrieves the X-Frame-Options value from the
        HTTP header. - More Info:
        http://msdn.microsoft.com/en-us/library/cc288472%28VS.85%29.
        aspx

        String       : SAMEORIGIN

[ X-XSS-Protection ]
        This plugin retrieves the X-XSS-Protection value from the
        HTTP header. - More Info:
        http://msdn.microsoft.com/en-us/library/cc288472%28VS.85%29.
        aspx

        String       : 0

HTTP Headers:
        HTTP/1.1 200 OK
        Date: Mon, 04 May 2021 16:09:47 GMT
        Expires: -1
        Cache-Control: private, max-age=0
        Content-Type: text/html; charset=ISO-8859-1
        P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
        Content-Encoding: gzip
        Server: gws
        Content-Length: 5762
        X-XSS-Protection: 0
        X-Frame-Options: SAMEORIGIN
        Set-Cookie: 1P_JAR=2021-05-04-16; expires=Wed, 03-Jun-2021 16:09:47 GMT; path=/; domain=.google.com; Secure
        Set-Cookie: NID=203=fEPs6hoaoVclld1HAxHMHF2N4gT5yNy6kBVL-abAzpg1fqqB4Yk4PJGOgdveogjY_ThytkYwmmLc4oVHA95jaQRujiByZ96QiPkheajk5hkPzL1LZyCF5kX_L3uHnLi9H9JpbEYk6FrjPNWMIOZjANYd7fpPaAj1emJPDwc-Clk; expires=Tue, 03-Nov-2021 16:09:47 GMT; path=/; domain=.google.com; HttpOnly
        Connection: close

This output is not only more detailed and easier to read but also more useful if being utilized in a script. We can also save the output to a file. To log the shorter output, use the --log-brief option, followed by the name of the file to write to:

~# whatweb google.com --log-brief brief.txt

/usr/lib/ruby/vendor_ruby/target.rb:188: warning: URI.escape is obsolete
/usr/lib/ruby/vendor_ruby/target.rb:188: warning: URI.escape is obsolete
http://google.com [301 Moved Permanently] Country[UNITED STATES][US], HTTPServer[gws], IP[172.217.8.174], RedirectLocation[http://www.google.com/], Title[301 Moved], X-Frame-Options[SAMEORIGIN], X-XSS-Protection[0]
http://www.google.com/ [200 OK] Cookies[1P_JAR,NID], Country[UNITED STATES][US], HTML5, HTTPServer[gws], HttpOnly[NID], IP[172.217.4.36], Script, Title[Google], X-Frame-Options[SAMEORIGIN], X-XSS-Protection[0]

And to log the verbose output, use the --log-verbose option:

~# whatweb google.com --log-verbose verbose.txt

/usr/lib/ruby/vendor_ruby/target.rb:188: warning: URI.escape is obsolete
/usr/lib/ruby/vendor_ruby/target.rb:188: warning: URI.escape is obsolete
http://google.com [301 Moved Permanently] Country[UNITED STATES][US], HTTPServer[gws], IP[172.217.6.14], RedirectLocation[http://www.google.com/], Title[301 Moved], X-Frame-Options[SAMEORIGIN], X-XSS-Protection[0]
http://www.google.com/ [200 OK] Cookies[1P_JAR,NID], Country[UNITED STATES][US], HTML5, HTTPServer[gws], HttpOnly[NID], IP[172.217.4.36], Script, Title[Google], X-Frame-Options[SAMEORIGIN], X-XSS-Protection[0]

Method 3: Wappalyzer

The next fingerprinting method we'll use is Wappalyzer, a browser extension that identifies the technologies a website uses when you visit the page. It's an extremely easy way to gather information about a target while manually enumerating webpages.

To get Wappalyzer, navigate to the extensions manager in Firefox and search for it in the search bar:

Next, click the "Add to Firefox" button:

And accept the permission requirements:

A notification will pop up confirming that Wappalyzer was added to Firefox — make sure to check the "Allow this extension to run in Private Windows" box:

Now that it's successfully installed, it redirects us to the Wappalyzer home page. You'll now notice a new icon on the right-hand side of the URL bar:

When we browse to a website, we can click on that icon to display information about the webpage and server:

Method 4: Online

The final method we'll use to fingerprint web apps and servers is arguably the easiest — we can do it entirely online. All we have to do on these sites is supply the website or host and all the technologies and frameworks in use will be identified.

The first site we will use is BuiltWith:

Simply enter the target and it will create a technology profile for us:

Scrolling down we can see some of the technologies in use:

The next fingerprinting site we'll look at is W3Techs:

Again, just enter a URL and it will give us an overview of the web technologies in use:

W3Techs even includes some visitor location stats, and if we scroll down we can see more detailed information:

Wrapping Up

Today, we learned about some of the popular web technologies and frameworks and how to fingerprint them to aid in reconnaissance. First, we used Netcat to connect via HTTP and grab information about the server. Next, we explored WhatWeb, a tool specifically designed to perform fingerprinting. After that, we covered some online options, including the Wappalyzer browser extension and a couple of sites that automatically identify web technologies.