News: Anonymity, Darknets and Staying Out of Federal Custody, Part One: Deep Web

Anonymity, Darknets and Staying Out of Federal Custody, Part One: Deep Web

Anonymity, Darknets and Staying Out of Federal Custody, Part One: Deep Web

You've probably seen those deep-web images floating around on the Internet. Usually, it goes something like this: There is a towering iceberg and the deeper the underwater portion extends, the more "hidden" and "exotic" the content is described to be. Sometimes these images are accurate to a point, but most are just making things up.

Anonymity, Darknets and Staying Out of Federal Custody, Part One: Deep Web

So what exactly is 'deep web' then? Are there really hidden secrets and treasure buried under some cloak and dagger type conspiracy? Well, in short, the answer depends on your idea of treasure and conspiracy.

In this series of articles, I am going to break down the idea of a deep web, what is it, how it got there, and most importantly, how we can use it for our security—maybe even for lulz.

Anonymity and Darknets

As it stands today, practical knowledge of how darknets (non-indexed portions of the Internet) function will allow you to make more informed decisions regarding risks when rooting a box, hiding files, or communicating securely. Now, keep in mind that nothing short of unplugging your computer will make you 100% anonymous. You can have fifty proxies and a handful of VPNs, but never consider yourself to be completely masked. Look at anonymity as a trade off between function and speed.

The idea is to have enough masking, while maintaining a level of usefulness. While it is not impossible to track your actions, ideally you want to make it logistically too complex to be attempted in a realistic way.

If you need to escape your school's firewall, a simple HTTP proxy may work for this. If you are rooting an AT&T server, you will want several layers between yourself and the target. Cyber crime laws are changing rapidly around the world, from Cairo to Chicago, and learning how to blend in with the masses is a valuable skill to have. You are harder to track when you are nobody at all.

First though, we need to talk about Google. Yes Google.

Why Google?

When you use Google to search, it does not take your query and search the entire Internet for results; there is simply too much data for that. It runs your search on databases of sites that have already been located by Google's web crawlers. These crawlers are bots, coded to search for, find, and index content on the web. Primarily, this is achieved by 'seeding' the crawler with a few initial links to start with. It scans for more hyperlinks on those websites, connecting to and repeating this process over and over while creating a 'map' of the results.

Anonymity, Darknets and Staying Out of Federal Custody, Part One: Deep Web

It is this 'map' of collected links that your search request is actually looking at. While this is innovative, it contains a few inherent flaws.

The Internet is large. In fact, the Internet is very large, and estimates on just how much of it is actually indexed and publicly searchable range from 40 to 70 percent. Problems arise in the fact that most search engines do not crawl through non-HTTP protocols like Gopher or FTP. And if they choose to, developers can take steps to minimize indexing of their sites altogether (controls like the robots.txt standard and spider traps are commonly used). It is worth noting that network resources requiring authorization are not crawled, under normal circumstances.

The surface web makes up over 90 percent of what you use and do online. The remaining network services require you to directly connect to them, log in to them, or otherwise know they are there beforehand.

Anonymity, Darknets and Staying Out of Federal Custody, Part One: Deep Web

So, what good is all of this for you?

Right now, not as much as you might think. Though the content might not be searched, and is sometimes exciting and risque, you are still held to the basic laws of the Internet, TCP/IP.

Every packet you send that zips back and forth has your IP address inside. It has to have your IP address, or the remote server would not know where to route the requests back to. This means that even if you are snooping around where you shouldn't be, even if it's not on an indexed site, those server logs can still give you up, even when a normal HTTP/SOCKS proxy is used. When your door gets kicked open and the Feds storm your living room, you will have wished you took the time to truly hide yourself.

Picture the Internet like a city, with each building as a resource with an address. Envision the non-indexed parts as alleyways, still connected to the main streets, but lacking public addresses of their own.

Anonymity, Darknets and Staying Out of Federal Custody, Part One: Deep Web

Let's take it one step farther... what if these alleyways had gates? What if you could create a path through the city using just the alley and your own private keys? You would be much harder to locate and follow. You could always pop out into the city, go into a house as you needed; people would only see you come and go from the alley, would not know where you started nor where you intend to go.

This is the basic idea behind low-latency anonymous networks like Tor and i2p. We will go over both of these in more detail, including installation and configuration, in the upcoming articles, so stay tuned.

UPDATE: Continue to Part Two: Onions and Daggers

Photos by TheNewsJunkie, Wikipedia, OSTI, eschipul, Ubertech

17 Comments

Like the analogies! Look forward to the continuation of this series!

This is great! awesome writing, rich in information :) much props.

Excellent introduction. Looking forward to part 2!

Really great post, I also love the analogies!

This gave me chills! Even though I know the article already, it was wow, very enlightening, easy to understand (still).
I actually looked for part two. I wanted to continue and learn more!
The image you gave with the words, to help us understand is extremely effective.
I came out of that completely understanding everything you said.
I now know why i2p and tor are needed and slightly what they do/ how they operate.

Great job man!

Great Article!

Random thought: What made you want encourage people to evade the "feds." Seeing as you used to work for the feds? jw. Great article. look forward to part 2. (though i really have no use for tor , except using facebook)

I am a believer in free speech. I think people should have the right to say whatever they wish without the govt coming down on you. If that sadly means this must happen on the deepweb, then so be it.

I can't want to read more for this. This was an amazing explanation. I'm a complete newb here, my first day in my journey to learning about social engineering/hacking/web anon and the like.
Thank you for a wonderful intro. =D

Thank you for your kind words. If you, or anyone else has any questions please let me know!

Hey, thanks,,, with the feds thinking the states don't have the right to dump the poisonpillpharma medications for true cures with medical marijuana,,, it seems most of us patients getting healthy need to keep anonymous to stay out of the jails that only have poison pills....I was on 7 meds several times a day = stayed in bed, praying for death. Started mmj and have been able to function, on the way to ' healthy ' ...to go back to the liver eating kidney killing pills would kill me fast.

I think the deep web is one of the few places free expression remains.

wow! nice idea about Deep web with the simplest example. I recently saw a book about deep web by an Italian Security enthusiastic Pierluigi Paganini . I just went through some pages of it he didn't provide simpler than you,you wrote very nice and in brief about it... thanxx Freeman ..

what is the best search engine?

well, recently a search engine came to light "Grams"... Its Tor link is grams7enufi7jmdl.onion . I've not checked it yet but it serves result from 8 online markets which include SilkRoad2, Agora, BlackBank, Cloud-Nine, Evolution, NiceGuy, Pandora, and The Pirate Market.

Share Your Thoughts

  • Hot
  • Latest