Hack Like a Pro: Metasploit for the Aspiring Hacker, Part 10 (Finding Deleted Webpages)

Metasploit for the Aspiring Hacker, Part 10 (Finding Deleted Webpages)

Welcome back, my greenhorn hackers!

Throughout this series on Metasploit, and in most of my hacking tutorials here on Null Byte that use Metasploit (there are many; type "metasploit" into the search bar and you will find dozens), I have focused primarily on just two types of modules: exploits and payloads. Remember, Metasploit has six types of modules:

  1. Exploit
  2. Payload
  3. Auxiliary
  4. NOP (no operation)
  5. Post (post exploitation)
  6. Encoder

Although most hackers and pentesters focus on the exploits and payloads, there is significant capability in the auxiliary modules that is often overlooked and ignored. About one-third of all of Metasploit—measured by lines of code—is auxiliary modules. These auxiliary modules encompass the capabilities of many other tools that a hacker requires, and include various types of scanners (including Nmap), denial-of-service modules, fake servers for capturing (admin) credentials, fuzzers, and many more.

In this tutorial, I want to explore and illuminate one of the auxiliary modules that can make hacking with Metasploit much more effective and efficient.

Step 1: List the Modules

First, let's fire up Kali and open a terminal. In Kali, Metasploit modules are stored at:


Let's navigate there:

kali > cd /usr/share/metasploit-framework/modules/

Then, let's list the contents of that directory:

kali > ls -l

As you can see, there are six types of modules in Metasploit, as mentioned before. Let's focus our attention on the auxiliary modules.

Step 2: List the Auxiliary Modules

First, navigate to the auxiliary module directory and list its contents:

kali > cd auxiliary

kali > ls -l

As you can see, there are numerous subdirectories of auxiliary modules. In an earlier tutorial, I pointed out that there are hundreds of auxiliary modules for DoSing in Metasploit. In this tutorial, we want to work with the scanner modules. Metasploit has scanning modules of just about every type, including Nmap and website vulnerability scanning.

Step 3: The Wayback Machine

As many of you know, recent years have seen an increased emphasis on information security. Webmasters and IT security personnel are more vigilant about what goes onto their website, trying to make certain that information that might be used to compromise their security is not posted.

If we go back just a few years, that was not the case. Companies often would post email addresses, passwords, vulnerability scans, network diagrams, etc. on their website, not aware that someone might find these and use them for malicious purposes. Although I still occassional find a company listing email addresses and passwords on their website, this has become much less common.

Fortunately for us, nothing ever disappears from the web. If it's here today, it will be here 20 years from now (keep that in mind when posting on Facebook or other social media sites).

The Internet Archive (Archive.org) was established to save free books, movies, music, software, along with all of the old web information via its Wayback Machine tool (which is a reference to an old cartoon where the main characters, Mr. Peabody and Sherman, would travel back in time in a time machine they called the "WABAC Machine"). This means that if a company had, at one time, stored email addresses and passwords on their webpages, or Nessus scans, it is still around somewhere on Archive.org.

Step 4: Use Metasploit to Retrieve Deleted Webpages

Fortunately for us, Metasploit has an auxiliary module that is capable of retrieving all of the old URLs from Archive.org that are stored for a particular domain. Since the Internet Archive's website tool is referred to as the Wayback Machine, Metasploit has a module called enum_wayback, short for "enumerate wayback" machine.

Let's start the Metasploit console and load it:

kali > msfconsole

When the msfconsole opens, let's load the wayback module by typing:

msf > use auxiliary/scanner/http/enum_wayback

This module basically has just two parameters to set:

  1. The domain we want to search for on archive.org
  2. The file we want to save the information in

Since we will be using the SANS.org as our target, let's set the output to a file named sans_wayback:

msf > set OUTFILE sans_wayback

Next, let's set the domain to our favorite IT Security domain:

msf > set DOMAIN sans.org

Step 5: Start the Wayback Machine

Unlike exploits in Metasploit where we type exploit to start them, auxiliary modules are initiated by typing run instead:

msf > run

This module will now go to Archive.org and begin to retrieve every saved URL of SANS.org over the years. In the case of SANS.org, it is over 96,000 URLs! Since we told this module to store all of the URLs in a file named "sans_wayback," all these URLs are written to this file.

When the module is done running, we can look inside this file by typing:

kali > more sans_wayback

When we do, we can see that Archive.org has stored the first URL from 1998. With over 90,000 URLs, visual inspection looking for interesting information is not really practical. Fortunately, god gave us the grep command.

If we want to find the URLs with emails stored in them, we can type:

kali > more sans_wayback | grep email

As you can see in the screenshot above, there are quite a few URLs pertaining to email. The one I have circled in red looks particularly interesting, email_list.htm.

It is not unheard of for companies to put their vulnerability scans on their website for the security and network personnel to view (and the rest of us). As Nessus is the world's most widely used vulnerability scanner, let's see whether we can find it among these old webpages.

kali > more sans_wayback | grep nessus

As you can see above, we have found three webpages with Nessus in them in the "audit" directory. Hmmm... that might be interesting... maybe some Nessus scan reports from the past?

This approach to finding deleted information is limited to the information in the URL. We could actually take these URLs and view them in our browser at Archive.org to see whether we can find text information on the page that might be useful. Maybe a better approach would be to download the interesting URLs that this module enumerated directly to a hard drive using HTTrack, then do a text search on the entire web content.

Keep coming back, my greenhorn hackers, as we explore the workings of the hacker's best friend, Metasploit!

Just updated your iPhone? You'll find new features for Podcasts, News, Books, and TV, as well as important security improvements and fresh wallpapers. Find out what's new and changed on your iPhone with the iOS 17.5 update.

Cover image via Shutterstock


Reading a new OTW post is like getting the next book to a favorite book series
Thanks OTW!

Also, where is Nessus located in Kali Linux?

(another thing, I plan to upgrade my kali usb to KaliSana, would you recommend the new Kali versus the old Kali? I heard KaliSana is faster, but is more buggy)

Nessus is not in Kali but you can download at tenable.com.

I prefer Kali 1.1.

Is there a way for Kali to keep Nessus even if I unplug the USB? because right now, (I run Kali on a liveUSB, boot into 648pae (I'm not sure what the other modes do yet), then when I shut down the computer and unplug the USB, everything I wrote is gone). Is there a way for Kali to keep Nessus even after I unplug the usb?

To keep information persistent on a USB when using Kali you will need to partition your USB, then boot to the option: "USB with persistence".

I have a usb drive with all of its space available for Kali
So would I still need to wipe the drive, re-create usb, then add Nessus?

Woah this is neat. I didnt even know this was a module. Thank you for the tutorial.

This will be very useful, thanks OTW

A very ingenious way to use the wayback machine!


Should i use this method for HTTPS protocol

trying it out. thanks

Hi there: What is the difference between using Metasploit Vs. using the search box at archive.org ? or searching Google Advanced search ?

I mean, how can Metasploit find a page that Google nor its own search engine can't find ?

Thank you


Mr.OTW you said that in auxiliaries unlike exploits and payloads and other modules we type "run" to initiate the module. However, you have clearly used the "exploit" command to initiate auxiliaries on other posts/articles. An example would be with the "setting up fake SMB servers" guide: "use auxiliary/server/capture/smb" ...the link is below.


Just found out that "exploit" also works. So basically it didn't matter....you are basically confusing people when you say things like that. Still appreciate what you have done. Idk if you're still alive on these old posts, but I have been studying the material you've been putting out rigorously. SO THANKS MR.OTW!!

Share Your Thoughts

  • Hot
  • Latest