While there are completely legitimate reasons to use Bitcoin, it's also used by terrorists, drug dealers, and other shady people that need to be investigated. That's where SpiderFoot comes in, which has a command-line interface to search for Bitcoin wallet addresses on a website and query the balances associated with them.
SpiderFoot is a great tool overall for automating OSINT (open-source intelligence), and there are two different versions: the free open-source project and SpiderFoot HX, a paid service that costs almost $800 a year. For our use, the free version is adequate for investigating Bitcoin wallets and balances, and we won't have to run a web server to do it, which is necessary if using SpiderFoot outside of the CLI.
Let's say an organization is raising funds using Bitcoin. Whether it's a legitimate or illegitimate organization that's asking for donations, you can monitor its performance by first locating any Bitcoin wallet addresses associated with the organization's domain by scraping the website. We can then pass that data into a query to find out the exact balance in each public-facing wallet.
Requirements
To use SpiderFoot's CLI, you'll need to have Python 3 installed, which comes preinstalled on newer Linux, macOS, and Windows systems. If it's not, or you have an older version of Python, Python 3 is easy enough to download and install.
Step 1: Install SpiderFoot
We're not going to go to SpiderFoot's website to get the tool. Instead, we'll get it from its GitHub repository. There are many good uses for SpiderFoot beyond what we're doing here today, so make sure to check out its GitHub to see everything else that it can do. It also has a list of modules that SpiderFoot can use, which can be combined and chained together to perform very specific search queries.
We'll be using two modules later on: one to search the website for the Bitcoin addresses, and the other to pass those addresses on to the balance query. But first, to download SpiderFoot via git, issue the following clone command in a terminal.
~$ git clone https://github.com/smicallef/spiderfoot.git
Cloning into 'spiderfoot'...
remote: Enumerating objects: 43, done.
remote: Counting objects: 100% (43/43), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 20781 (delta 17), reused 18 (delta 7), pack-reused 20738
Receiving objects: 100% (20781/20781), 13.89 MiB | 9.34 MiB/s, done.
Resolving deltas: 100% (16727/16727), done.Then, change into its directory and perform a listing (ls).
~$ cd spiderfoot
~/spiderfoot$ ls
dicts modules sfcli.py static
Dockerfile passwd sflib.py test
dyn README.md sf.py THANKYOU
generate-certificate requirements_test.txt sfscan.py VERSION
LICENSE requirements.txt sfwebui.py
log setup.cfg spiderfootYou can see a requirements.txt file, so make sure to install that with pip3 before continuing. If you don't, chances are that SpiderFoot will fail because it doesn't have all the dependencies it needs to function correctly.
~/spiderfoot$ pip3 install -r requirements.txt
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: adblockparser>=0.7 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 1)) (0.7)
Requirement already satisfied: dnspython>=1.16.0 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 2)) (2.0.0)
Requirement already satisfied: exifread>=2.1.2 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 3)) (2.3.2)
Requirement already satisfied: CherryPy>=18.0 in /home/kali/.local/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (18.6.0)
Requirement already satisfied: cherrypy-cors>=1.6 in /home/kali/.local/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (1.6)
Requirement already satisfied: Mako>=1.0.4 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 6)) (1.1.3)
Requirement already satisfied: beautifulsoup4>=4.4.1 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 7)) (4.9.3)
Requirement already satisfied: lxml>=4.6.1 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 8)) (4.6.1)
Requirement already satisfied: netaddr>=0.7.18 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 9)) (0.7.19)
Requirement already satisfied: pysocks>=1.7.1 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 10)) (1.7.1)
Requirement already satisfied: requests>=2.20.0 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 11)) (2.24.0)
Requirement already satisfied: ipwhois==1.0.0 in /home/kali/.local/lib/python3.8/site-packages (from -r requirements.txt (line 12)) (1.0.0)
Requirement already satisfied: ipaddr>=2.2.0 in /home/kali/.local/lib/python3.8/site-packages (from -r requirements.txt (line 13)) (2.2.0)
Requirement already satisfied: phonenumbers>=8.12.9 in /home/kali/.local/lib/python3.8/site-packages (from -r requirements.txt (line 14)) (8.12.13)
Requirement already satisfied: pygexf>=0.2.2 in /home/kali/.local/lib/python3.8/site-packages (from -r requirements.txt (line 15)) (0.2.2)
Requirement already satisfied: PyPDF2>=1.26.0 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 16)) (1.26.0)
Requirement already satisfied: stem>=1.7.1 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 17)) (1.8.0)
Requirement already satisfied: python-whois>=0.7.1 in /home/kali/.local/lib/python3.8/site-packages (from -r requirements.txt (line 18)) (0.7.3)
Requirement already satisfied: secure>=0.2.1 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 19)) (0.2.1)
Requirement already satisfied: pyOpenSSL>=17.5.0 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 20)) (19.1.0)
Requirement already satisfied: python-docx>=0.8.10 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 21)) (0.8.10)
Requirement already satisfied: python-pptx>=0.6.18 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 22)) (0.6.18)
Requirement already satisfied: networkx>=2.5 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 23)) (2.5)
Requirement already satisfied: cryptography>=3.2.1 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 24)) (3.2.1)
Requirement already satisfied: publicsuffixlist>=0.7.3 in /home/kali/.local/lib/python3.8/site-packages (from -r requirements.txt (line 25)) (0.7.5)
Requirement already satisfied: portend>=2.1.1 in /home/kali/.local/lib/python3.8/site-packages (from CherryPy>=18.0->-r requirements.txt (line 4)) (2.7.0)
Requirement already satisfied: more-itertools in /usr/lib/python3/dist-packages (from CherryPy>=18.0->-r requirements.txt (line 4)) (4.2.0)
Requirement already satisfied: zc.lockfile in /home/kali/.local/lib/python3.8/site-packages (from CherryPy>=18.0->-r requirements.txt (line 4)) (2.0)
Requirement already satisfied: cheroot>=8.2.1 in /home/kali/.local/lib/python3.8/site-packages (from CherryPy>=18.0->-r requirements.txt (line 4)) (8.4.7)
Requirement already satisfied: jaraco.collections in /home/kali/.local/lib/python3.8/site-packages (from CherryPy>=18.0->-r requirements.txt (line 4)) (3.0.0)
Requirement already satisfied: httpagentparser>=1.5 in /home/kali/.local/lib/python3.8/site-packages (from cherrypy-cors>=1.6->-r requirements.txt (line 5)) (1.9.0)
Requirement already satisfied: soupsieve>1.2 in /usr/lib/python3/dist-packages (from beautifulsoup4>=4.4.1->-r requirements.txt (line 7)) (2.0.1)
Requirement already satisfied: future in /usr/lib/python3/dist-packages (from python-whois>=0.7.1->-r requirements.txt (line 18)) (0.18.2)
Requirement already satisfied: tempora>=1.8 in /home/kali/.local/lib/python3.8/site-packages (from portend>=2.1.1->CherryPy>=18.0->-r requirements.txt (line 4)) (4.0.1)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from zc.lockfile->CherryPy>=18.0->-r requirements.txt (line 4)) (50.3.0)
Requirement already satisfied: jaraco.functools in /home/kali/.local/lib/python3.8/site-packages (from cheroot>=8.2.1->CherryPy>=18.0->-r requirements.txt (line 4)) (3.0.1)
Requirement already satisfied: six>=1.11.0 in /usr/lib/python3/dist-packages (from cheroot>=8.2.1->CherryPy>=18.0->-r requirements.txt (line 4)) (1.15.0)
Requirement already satisfied: jaraco.text in /home/kali/.local/lib/python3.8/site-packages (from jaraco.collections->CherryPy>=18.0->-r requirements.txt (line 4)) (3.2.0)
Requirement already satisfied: jaraco.classes in /home/kali/.local/lib/python3.8/site-packages (from jaraco.collections->CherryPy>=18.0->-r requirements.txt (line 4)) (3.1.0)
Requirement already satisfied: pytz in /usr/lib/python3/dist-packages (from tempora>=1.8->portend>=2.1.1->CherryPy>=18.0->-r requirements.txt (line 4)) (2020.4)If you're having issues installing it via the command above, try using apt update first, then try again. If you still have issues, run the command with sudo in front of it.
Step 2: Make Sure SpiderFoot Is Working
Now that SpiderFoot is all ready to go, it's time to start using it with a command-line argument to pass the information we'd like to have returned. From the listing performed above, you'll see the sf.py file, and that's the main program we'll be using here. Let's try running it to see if it's working:
~/spiderfoot$ python3 ./sf.py
You must specify a target when running in scan mode. Try --help for guidance.If you see something else, or if you run into errors now or in a later step, go back and reinstall the requirements.txt with pip3. If that doesn't work, install it with sudo, which should fix any missing modules.
Now, let's quickly review the help page to see the usage instructions.
~/spiderfoot$ python3 ./sfcli.py --help
usage: sf.py [-h] [-d] [-l IP:port] [-m mod1,mod2,...] [-M] [-s TARGET]
[-t type1,type2,...] [-T] [-o tab|csv|json] [-n] [-r] [-S LENGTH]
[-D DELIMITER] [-f] [-F FILTER] [-x] [-q]
SpiderFoot 3.0: Open Source Intelligence Automation.
optional arguments:
-h, --help show this help message and exit
-d, --debug Enable debug output.
-l IP:port IP and port to listen on.
-m mod1,mod2,... Modules to enable.
-M, --modules List available modules.
-s TARGET Target for the scan.
-t type1,type2,... Event types to collect.
-T, --types List available event types.
-o tab|csv|json Output format. Tab is default.
-n Strip newlines from data.
-r Include the source data field in tab/csv output.
-S LENGTH Maximum data length to display. By default, all data is
shown.
-D DELIMITER Delimiter to use for CSV output. Default is ,.
-f Filter out other event types that weren't requested with
-t.
-F FILTER Filter out a set of event types.
-x STRICT MODE. Will only enable modules that can directly
consume your target, and if -t was specified only those
events will be consumed by modules. This overrides -t
and -m options.
-q Disable logging.Step 3: Scan for Bitcoin Addresses & Balances
Now, let's look at a Bitcoin address and wallet balance that we scrap from a website. We're going to have to chain a couple of things together to do it. And this is where the modules come into play.
~/spiderfoot$ python3 ./sf.py -m sfp_spider,sfp_bitcoin,sfp_blockchain -s websiteurl.com -F -q BITCOIN_ADDRESS,BITCOIN_BALANCE -qAbove, you can see the command we're using to do what we need. The first part of the command is starting the SpiderFoot tool with Python 3. The -m indicates that you're using a module or modules, and right after it, that's where the modules go, separated by commas.
The sfp_spider module will spider the website's webpages we're scanning to find all of the information inside. The sfp_bitcoin module will isolate all of the Bitcoin wallet addresses that it finds via the spidering. And the sfp_blockchain module will take the Bitcoin address or addresses and pass it or them along to an API that will determine the exact amount stored in the wallet.
Next in the command, we have -s, followed by the website URL, which will let us pick the target website. Then, we have -F to filter out a set of event types, specifically, BITCOIN_ADDRESS and BITCOIN_BALANCE. That's the information we want to find. Finally, -q is used to hide everything in the output except only the stuff we want.
Now, let's run it against a real website, using bitcoinforcharity.com as an example.
~/spiderfoot$ python3 ./sf.py -m sfp_spider,sfp_bitcoin,sfp_blockchain -s bitcoinforcharity.com -F BITCOIN_ADDRESS,BITCOIN_BALANCE -q
Source Type Data
sfp_bitcoin Bitcoin Address 1HesYJSP1QqcyPEjnQ9vzBL1wujruNGe7R
sfp_blockchain Bitcoin Balance 0.00021 BTC
sfp_bitcoin Bitcoin Address 16Sy8mvjyNgCRYS14m1Rtca3UfrFPzz9eJ
sfp_blockchain Bitcoin Balance 0.24481116 BTC
sfp_bitcoin Bitcoin Address 1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd
sfp_blockchain Bitcoin Balance 1.62908644 BTC
sfp_bitcoin Bitcoin Address 1946W6LDsEYF9B5sPYDKfwLw6YBZuHns4L
sfp_blockchain Bitcoin Balance 0.02344126 BTC
sfp_bitcoin Bitcoin Address 1PC9aZC4hNX2rmmrt7uHTfYAS3hRbph4UN
sfp_blockchain Bitcoin Balance 1.89378293 BTC
sfp_bitcoin Bitcoin Address 1PAt5oKQGBRigFDY6fB2WgQTtQJNzFyTDr
sfp_blockchain Bitcoin Balance 0.0 BTC
sfp_bitcoin Bitcoin Address 1NgiUwkhYVYMy3eoMC9dHcvdHejGxcuaWm
sfp_blockchain Bitcoin Balance 0.06070947 BTC
sfp_bitcoin Bitcoin Address 1M87hiTAa49enJKVeT9gzLjYmJoYh9V98
sfp_blockchain Bitcoin Balance 0.0 BTC
sfp_bitcoin Bitcoin Address 1CU5YgjquupDw6UeXEyA9VEBH34R7fZ19b
sfp_blockchain Bitcoin Balance 0.16549195 BTC
sfp_bitcoin Bitcoin Address 16DEzKc9fX4XfgGzEvQUJmoYeUrbRNXqxe
sfp_blockchain Bitcoin Balance 0.18967667 BTC
sfp_bitcoin Bitcoin Address 1AS3TiTqgJZK6CfNfqcbPXSx4PTFvfghvF
sfp_blockchain Bitcoin Balance 0.0 BTC
sfp_bitcoin Bitcoin Address 1Archive1n2C579dMsAu3iC6tWzuQJz8dN
sfp_blockchain Bitcoin Balance 3.17865301 BTCFrom the listed output above, we can see that the charity has more than a few publicly listed Bitcoin addresses leading back to wallets where we see the exact bitcoin amount. The charity above isn't really raising any real bitcoin, with 3.17865301 BTC being the wallet with the most. That's not particularly interesting if we were looking for a site moving lots of money through cryptocurrencies.
Step 4: Scan for Ethereum Addresses
For Ethereum, you could use a different command structure to get all of the wallet addresses found on a website, but you won't be able to get any balances as you could with Bitcoin. To demonstrate quickly, we'll scan etherdonation.com.
~/spiderfoot$ python3 ./sf.py -m sfp_spider,sfp_ethereum -s etherdonation.com -F ETHEREUM_ADDRESS -q
Source Type Data
sfp_ethereum Ethereum Address 0xed6ca7d908f897d0b0d5f9b9e7aa470698e10b1b
sfp_ethereum Ethereum Address 0xed6ca7d908f897d0b0d5f9b9e7aa470698e10b1b
sfp_ethereum Ethereum Address 0xed6ca7d908f897d0b0d5f9b9e7aa470698e10b1b
sfp_ethereum Ethereum Address 0xed6ca7d908f897d0b0d5f9b9e7aa470698e10b1b
sfp_ethereum Ethereum Address 0xed6ca7d908f897d0b0d5f9b9e7aa470698e10b1b
sfp_ethereum Ethereum Address 0xed6ca7d908f897d0b0d5f9b9e7aa470698e10b1bAll we did there was use sfp_spider, the spidering module, and sfp_ethereum, the Ethereum finding module. And we removed the filter for the balance since we can't find Ethereum balances.
Simple but Powerful
If you want to see some of the organizations with the largest bitcoin transactions, then you can google something like "top 100 bitcoin addresses" and pick a result that monitors the top ones. Then, you can use that site's URL in the command to see which Bitcoin addresses are moving the most BTC.
Just updated your iPhone? You'll find new emoji, enhanced security, podcast transcripts, Apple Cash virtual numbers, and other useful features. There are even new additions hidden within Safari. Find out what's new and changed on your iPhone with the iOS 17.4 update.
2 Comments
awesome
Hi tried entering the command pip3 install -r require.txt but it doesn't work. I tried all the other alternatives you gave but to no avail. I'm using the git terminal for commands and I'm not sure if Python 3 is causing the problem but I already have it installed
Share Your Thoughts