How To: Write Your Own Subdomain Enumeration Script for Better Recon

There are tons of tools out there that do all kinds of recon, but it can be hard to narrow down what to use. A great way to be more efficient is by taking advantage of scripting. This doesn't have to mean writing everything from scratch — it can simply mean integrating existing tools into a single, comprehensive script. Luckily, it's easy to create your own subdomain enumeration script for better recon.

Step 1: Install Dependencies

Before we begin, there are a few things we need to install and set up for everything to work properly. First, make sure Go and Subfinder are installed on the system. Second, we'll be using a tool called assetfinder for additional subdomain recon; we can get the latest release from GitHub with:

~# wget https://github.com/tomnomnom/assetfinder/releases/download/v0.1.0/assetfinder-linux-amd64-0.1.0.tgz

--2021-04-28 15:00:12--  https://github.com/tomnomnom/assetfinder/releases/download/v0.1.0/assetfinder-linux-amd64-0.1.0.tgz
Resolving github.com (github.com)... 140.82.114.4
Connecting to github.com (github.com)|140.82.114.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/193392376/6e64a200-d33f-11e9-9d79-2165e6e68bb1?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210428T200012Z&X-Amz-Expires=300&X-Amz-Signature=3704ee96ec028f1ac8de3a3af870351ff434bdbd1150e3893a2cd02d43113b71&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=193392376&response-content-disposition=attachment%3B%20filename%3Dassetfinder-linux-amd64-0.1.0.tgz&response-content-type=application%2Foctet-stream [following]
--2021-04-28 15:00:12--  https://github-production-release-asset-2e65be.s3.amazonaws.com/193392376/6e64a200-d33f-11e9-9d79-2165e6e68bb1?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210428T200012Z&X-Amz-Expires=300&X-Amz-Signature=3704ee96ec028f1ac8de3a3af870351ff434bdbd1150e3893a2cd02d43113b71&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=193392376&response-content-disposition=attachment%3B%20filename%3Dassetfinder-linux-amd64-0.1.0.tgz&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.46.132
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.46.132|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3739744 (3.6M) [application/octet-stream]
Saving to: ‘assetfinder-linux-amd64-0.1.0.tgz’

assetfinder-linux-amd64-0.1.0.tgz         100%[=====================================================================================>]   3.57M  1.78MB/s    in 2.0s

2021-04-28 15:00:14 (1.78 MB/s) - ‘assetfinder-linux-amd64-0.1.0.tgz’ saved [3739744/3739744]

And use tar to extract the binary:

~# tar xzf assetfinder-linux-amd64-0.1.0.tgz

Then, move assetfinder to a directory in our path:

~# mv assetfinder /usr/local/bin/

Third, we need a tool called httprobe, which will allow us to filter live hosts in our results. Grab the release from GitHub with:

~# wget https://github.com/tomnomnom/httprobe/releases/download/v0.1.2/httprobe-linux-amd64-0.1.2.tgz

--2021-04-28 15:05:40--  https://github.com/tomnomnom/httprobe/releases/download/v0.1.2/httprobe-linux-amd64-0.1.2.tgz
Resolving github.com (github.com)... 140.82.114.4
Connecting to github.com (github.com)|140.82.114.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/80510806/d4c97700-afc2-11e9-9a18-8f50cc10ac23?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210428T200541Z&X-Amz-Expires=300&X-Amz-Signature=35781254f155f3fd67a026f17035c7fa9f0124feed26e08a305266c73eff08f0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=80510806&response-content-disposition=attachment%3B%20filename%3Dhttprobe-linux-amd64-0.1.2.tgz&response-content-type=application%2Foctet-stream [following]
--2021-04-28 15:05:41--  https://github-production-release-asset-2e65be.s3.amazonaws.com/80510806/d4c97700-afc2-11e9-9a18-8f50cc10ac23?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210428T200541Z&X-Amz-Expires=300&X-Amz-Signature=35781254f155f3fd67a026f17035c7fa9f0124feed26e08a305266c73eff08f0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=80510806&response-content-disposition=attachment%3B%20filename%3Dhttprobe-linux-amd64-0.1.2.tgz&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.44.212
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.44.212|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3555994 (3.4M) [application/octet-stream]
Saving to: ‘httprobe-linux-amd64-0.1.2.tgz’

httprobe-linux-amd64-0.1.2.tgz            100%[=====================================================================================>]   3.39M  1.61MB/s    in 2.1s

2021-04-28 15:05:43 (1.61 MB/s) - ‘httprobe-linux-amd64-0.1.2.tgz’ saved [3555994/3555994]

Unpack the binary:

~# tar xzf httprobe-linux-amd64-0.1.2.tgz

And move it to a directory in our path:

~# mv httprobe /usr/local/bin/

Fourth, we need to configure a couple of things for Go. First, make a directory called go:

~# mkdir /usr/local/go

Use the following command to set the GOPATH environmental variable:

~# go env -w GOPATH=/usr/local/go

We can confirm that we set it correctly:

~# go env GOPATH

/usr/local/go

Then, we need to add GOPATH to our path. Use the following command, creating the /bin directory if it doesn't already exist:

~# export PATH=$PATH:$(go env GOPATH)/bin

Next, we can make our changes persistent by adding the configuration to our .bashrc file:

~# echo 'export GOPATH=/usr/local/go' >> ~/.bashrc

Use the following command to source the file, making it persistent:

~# . ~/.bashrc

Last, we need a tool called Subjack; we'll get into what this tool does later on, but for now, we can install it with the go get command:

~# go get github.com/haccer/subjack

That will automatically install it in our GOPATH and make it ready to use. And that should be everything we need, so now let's start our script.

Step 2: Start the Script

To begin, create a script and open it with your favorite text editor:

~# nano subrecon.sh

The first line we need, called a shebang or hashbang, will point to the system's interpreter. This tells the system how to run the file; in this case, it is a Bash script:

#!/bin/bash

Next, we will make sure the user supplies input to the script, and if not, prints a usage example and exits. Use a conditional if-then block:

if [ -z $1 ]
then
        echo './subrecon.sh <list of domains>'
        exit 1
fi

The $1 is the argument passed to the script, and the -z option returns true if the string is null. So basically, this says if no argument is supplied, show the usage and exit. The argument we'll pass in is a list of domains.

• Don't Miss: How to Uncover Hidden Subdomains to Reveal Internal Services with CT-Exposer

Step 3: Enumerate Subdomains

The first action our script will take is enumerating subdomains:

echo 'FINDING SUBDOMAINS...'

while read $line
do
        for var in $line
        do
                echo 'enumerating:' $var

                subfinder -silent -d $var > out1
                cat out1 >> subs1

                assetfinder -subs-only $var > out2
                cat out2 >> subs2

                rm out1 out2
        done
done < $1

This will use a while loop to read input from our file containing a list of domains, use a variable to display the current domain being enumerated, and gather results from both Subfinder and assetfinder.

The next section will combine the results, remove any duplicates, and save the output to a file called all_subs:

sort -u subs1 subs2 > all_subs
rm subs1 subs2
echo 'saved subdomains to all_subs'

Step 4: Determine Live Hosts

The next part of the script will determine which hosts from the previous results are live. This is extremely useful for cutting down the time it takes to go through everything since hosts that are down are usually of no interest.

This will take the list of subdomains and use httprobe to filter out live hosts, saving the results to a file called live_subs:

echo 'FINDING LIVE HOSTS...'

cat all_subs | httprobe > live_subs
echo 'saved live hosts to live_subs'

Step 5: Test for Subdomain Takeover

Subdomain takeover is the process of registering a domain name to gain control over another domain. This happens when a host, usually a subdomain, points to a service that is no longer in use. The most common scenario is when a subdomain points to another domain, the DNS record expires, and the domain is available to be registered by someone else. Anyone who can successfully register the domain now has full control over the subdomain.

• Don't Miss: How to Quickly Look Up the Valid Subdomains for Any Website

In some cases, this type of attack is not possible due to verification methods, but you'd be surprised by how many services are vulnerable to subdomain takeover. Amazon S3 buckets, GitHub pages, Heroku, Shopify, and Microsoft Azure are all prone to this attack in some shape or form.

Subjack is a handy tool that will test a list of subdomains for potential takeover. Here, we will use the -w flag for an input file and the -a flag to send requests to every URL:

echo 'CHECKING FOR SUBDOMAIN TAKEOVER...'

subjack -w all_subs -a

echo 'DONE'

If anything in our list is vulnerable to subdomain takeover, the results will show on-screen along with the associated service.

Step 6: Review the Script

The final script should look something like this:

#!/bin/bash

if [ -z $1 ]
then
        echo './subrecon.sh <list of domains>'
        exit 1
fi

echo 'FINDING SUBDOMAINS...'

while read line
do
        for var in $line
        do
                echo 'enumerating:' $var

                subfinder -silent -d $var > out1
                cat out1 >> subs1

                assetfinder -subs-only $var > out2
                cat out2 >> subs2

                rm out1 out2
        done
done < $1

sort -u subs1 subs2 > all_subs
rm subs1 subs2
echo 'saved subdomains to all_subs'

echo 'FINDING LIVE HOSTS...'

cat all_subs | httprobe > live_subs
echo 'saved live hosts to live_subs'

echo 'CHECKING FOR SUBDOMAIN TAKEOVER...'

subjack -w all_subs -a

echo 'DONE'

Now it's time to test it out. Save the script, then make it executable:

~# chmod +x subrecon.sh

And run it, supplying a list of domains to enumerate:

~# ./subrecon.sh domains.txt

FINDING SUBDOMAINS...
enumerating: wonderhowto.com
saved subdomains to all_subs
FINDING LIVE HOSTS...
saved live hosts to live_subs
CHECKING FOR SUBDOMAIN TAKEOVER...
DONE

This is a good start, but the beauty of this script is that it can easily be expanded. Anything useful for recon, especially subdomain recon, can be added to make the process of enumeration unique.

Wrapping Up

In this tutorial, we learned how to write our own subdomain enumeration script in Bash. First, we installed some dependencies and got started on our script. Next, we used Subfinder and assetfinder to discover subdomains and combine the results, and filtered out live hosts with httprobe. Finally, we utilized Subjack to check for potential subdomain takeover.