The Deep Web is not accessible through search engines, but it comprises the majority of Internet traffic. Its variety of uses may surprise you.
n September 2014, the Spike Network announced plans to develop Deep Web, a television drama co-produced by Gary Oldman and Tony Krantz. Sharon Levy, a Spike Vice President, hailed the project as “wholly distinctive” and “ripped from today’s headlines.” In fact, the Deep Web TV project is ripped from one headline in particular: a November 2013 Time feature about the rise and fall of the Silk Road, the infamous online market best known for its thriving drug trade. Its operator, a straitlaced former Eagle Scout named Ross Ulbricht, assumes an alternate identity and builds an anonymous, multimillion-dollar marketplace for anything from heroin and GHB to guns and forgeries. The story unfolds like a parable of Libertarianism run amok, or the elaborate fantasy of any rule-abiding person who secretly dreams of getting rich by beating the system. But every detail — if federal prosecutors are correct — is true.
It would be easy to assume that a television pilot marks the death of what was once an underworld, particularly since we know how the story ends: In October 2013, Ulbricht was apprehended by the FBI and faces charges including conspiracy and attempted murder-for-hire. But over the course of that fall, Ulbricht’s arrest was just one of several busts targeting the illegal markets that had come to flourish on the Deep Web. During the months prior, the FBI had secretly taken control of Freedom Hosting, a major provider of anonymous web, email, and financial services that operated on the Deep Web. According to the Tor blog, a “large number” of hidden services were breached in the operation, including dozens dedicated to child porn. The FBI had planted bad JavaScript designed to collect the identities of Freedom Hosting administrators, eventually leading them to Eric Eoin Marques, a 28-year-old Irish citizen who allegedly “dove for his laptop” during the raid.
The FBI, however, wasn’t the first organization to target what was long suspected to be a linchpin of the child porn trade. In 2011, the hacktivist group Anonymous launched a campaign called #OPDarkNet against the operators of Lolita City, a notorious pedophile forum supported by Freedom Hosting. During the operation, Anonymous published screen names of 1,589 active users, and claimed responsibility for DDoS (Distributed Denial of Service) attacks against other sites tied to Freedom Hosting. Though the #OPDarkNet campaign was not successful in dismantling Freedom Hosting, it helped set the tone for self-regulation initiatives on the Darknet, a loosely collectivized effort by some site operators to shun the worst actors with the goal of preserving the whole.
Did these events kill, or even cripple the Darknet? Hardly. Despite high-profile stings, online vigilantism, and sensational Time stories, the hidden Internet persists while remaining largely misunderstood. Though “Deep Web” and “Darknet” are often used interchangeably, they refer to slightly different concepts in Internet research and mainstream parlance. The term “Deep Web” was coined in 2001 by Michael Bergman, whose research described search engines like Google, Yahoo, or (once-formidable) Lycos as ships running a dragnet over the surface of the ocean, leaving a vast bounty of data undiscovered by the automated web crawlers that bounce from site to site, indexing data for the engine’s searchable directories. Relying on crawlers meant that if a site or database is not directly linked to by another site, it will go undiscovered by crawlers. At the time, this included everything from members-only news sites to the then-nascent Amazon.com. Amazingly, Bergman found that Deep Web content was “400 to 550 times larger than the commonly defined World Wide Web.” Perhaps even more amazing, search engines still rely overwhelmingly on crawlers to index pages. This helps explain why 13 years later, Bergman’s original definition of the Deep Web — basically, whatever you can’t get to through Google — still resonates today.
Nowadays — thanks in large part to the well-publicized Silk Road takedown — the Deep Web evokes seedy underground markets peddling drugs, weapons, stolen goods, child porn, and worse. But the black markets active in recent years evolved alongside the Tor (The Onion Router) Project, an anonymity network originally funded by the U.S. Navy. First launched in 2002, the software was intended to protect the integrity of U.S. intelligence communications. By bouncing data packets through layers of randomized nodes — hence the onion moniker — Tor ensures the anonymity of users and web hosts. But, as made clear in the original 2004 paper outlining the technology, “a system with fewer users provides less anonymity. Usability is thus not only a convenience: it is a security requirement.” In other words, the Tor technology requires widespread adoption in order to work. So later that year, the U.S. Naval Research Lab released the project under an open-source license, making it available to anyone with an interest in staying anonymous online: military, whistle-blowers, journalists, dissidents, and anti-censorship activists. Throughout the Egyptian protests of 2011, for example — during which the government shut off access to the global Internet — Tor usage spiked within the country to 1.7 percent of all global users. Reporters Without Borders advises its journalists, bloggers, and their sources to use Tor in repressive environments. Tor has even been credited in helping victims of domestic abuse, who may suffer intense cyberstalking at the hands of their abusers. Tor’s diversity is reflected in its active sponsor base, including Google, Radio Free Asia, the National Science Foundation, the U.S. Department of State, and thousands of individual donors.
The problems that Tor creates for governments are genuine. The Russian government offered a bounty of 3.9 million ruble for technology that can identity Tor users en masse. And paradoxically enough, considering its origins in government research, the problems extend to the U.S. government as well. In the Snowden leaks, a powerpoint deck bluntly entitled “Tor Stinks” bemoaned the headaches that the technology had caused for intelligence analysts: “We will never be able to de-anonymize all Tor users all the time… (but) with manual analysis we can de-anonymize a very small fraction.” In the presentation, suggested counter-Tor measures include laborious manual efforts such as identifying “dumb” users likely to reveal their identities. The implication of the slides is clear: Tor is effective. If you use it properly, don’t act stupid, and follow the basic guidelines recommended by project’s administrators, you will almost assuredly remain anonymous.
It is a cardinal rule of the Internet, however, that where anonymity exists, assholes follow. (One need look no further than YouTube comment threads to observe this fundamental truth.) And even after the Freedom Hosting bust, the stain of child abuse is still present for Tor’s hidden services. Recently, I searched the term “hidden services” on Yahoo!, and an auto-generated advertisement for Ask.com appeared with a URL embedded: ask.com/hiddenhardcandy.
This would-be URL, generated from associated searches, does not actually exist. But it shows that one of the most common keywords associated with “hidden” is still Hard Candy, which was one of the child porn forums threatened by Anonymous in 2011. While the Deep Web can mean anything that Google or Yahoo! does not or cannot index, the Darknet more often describes the economies that inevitably form in those shadowy places that search engines and law enforcement can’t easily reach.
began using the Darknet in early 2013, while researching a story about the growing popularity of the Silk Road. A couple of weeks earlier, I had run into an acquaintance in a San Francisco park. My jollier-than-normal friend — a rather reserved software salesman by day — was tripping on mescaline, and freely shared his experiences buying hallucinogens on the Silk Road. “It’s fucking great,” he emphasized. “If you have Bitcoin, it’s actually easier to use than eBay.”
And by that point, it was. Earlier that year, a company called Coinbase had built the first Bitcoin exchange to cooperate with major banks, whereby you could create a Bitcoin wallet that linked directly to your checking account. For years, acquiring Bitcoin (BTC) was an arcane process involving “mining” — manually generating coins using custom software and lots of processing muscle — or individually coordinated, peer-to-peer trades. But by Spring 2013 this was no longer the case. Thanks to Coinbase, getting Bitcoin was now as easy as using PayPal. I bought a single Bitcoin for about $22 and downloaded the Tor software. Now a world of drugs, explosives, suspicious biotic material, hackers-for-hire, odd eBooks, hit men, disgusting porn, and wildly inexpensive designer apparel was available to me.
Likewise, locating the Silk Road was about as difficult as finding “banned” celebrity nudes: that is, incredibly easy if you’re willing to put in about 30 seconds’ worth of Googling. Tor works exactly like any other browser, except it can access both the surface web and the .onion domains that host the black markets, anonymous forums, and “hidden services” that the underground web is known for. The URLs for these sites are not broadly publicized. There is a Reddit page dedicated to re-posting addresses, but they frequently migrate. Yet a few searches will always turn up a list of active hidden sites, which include Agora Marketplace, Cloud Nine, Evolution, Tor Bazaar, and dozens of others. The majority of the commonly publicized sites are similar in layout, peddling a predictable inventory of drugs, fake IDs, credit card dumps, and a couple of pistols.
This wasn’t necessarily always the case. When I first visited the Silk Road, in its heyday, there were explosives, stolen credit cards, IDs and PayPal logins, manuals on everything from bomb making to identity theft, and an erotica section I never had the stomach to click on. On other hidden services, I found a hit man, a seller offering jars of “pregnant urine” for $100 a pop, and a crowd sourced assassination site that tried to place Bitcoin bounties on political targets.
Certainly, I was barely scratching the surface of what the Darknet had to offer. My liability-related paranoia (I was working for a large media company at the time) and aversion to things like violent porn interfered with the reach of my research. But just looking through the relatively vanilla “digital services” section, I was struck by just how little an identity is worth on the Darknet. Stolen credit card PINs are still offered by the thousands for less than $20 in BTC. Need to break into someone’s email or Facebook for a specific reason? No questions asked, but it will cost .1 BTC — curiously, far more expensive than violating dozens or even hundreds of identities at once. According to one hacker-for-hire I emailed, all he or she would need was the email address, the date of birth of the victim, and a couple of hours. I would send the coin, they would send the correct login information, and the transaction would be complete.
Vendors on the Darknet are pleasant but rightly paranoid, often requesting to communicate only through encrypted email services. At the time, I used Shazzlemail with a made-up identity. (Nowadays, most vendors will also ask that you use a PGP key, which adds an extra layer of encryption to email and other communications.) Too scared to order drugs or hacking services, I settled on a bootleg designer wallet and paid with about $5 worth of Bitcoin. The seller was as polite and responsive as any you’d find on eBay, and my package arrived a couple of business days later in a perfectly normal bubble-wrapped envelope. My Bitcoin was held in escrow until the package arrived, and I still use this wallet today. I never ordered anything from Silk Road again, but a year later, the remainder of the Bitcoin had ballooned in value to about $500. (I eventually sold it for several hundred dollars’ profit.)
As the mainstreaming of the Darknet continued, new marketplaces appeared, hoping to capture a new wave of users seeking easy access to drugs. One of these, a market called Atlantis, advertised itself with an oddly well-produced video ad, openly promoted on YouTube, which cheerfully introduced itself as a great place to buy weed should you have to move cities. Was Atlantis a honeypot for law enforcement? An entrepreneurial experiment? A pyramid scheme? That has never been established, but a few months later Atlantis announced on its Facebook page that unspecified “security reasons” had forced them to close, but that leftover Bitcoin would be donated to “a drug charity of their choice”.
The honeypot theory was a plausible one, since — like anyone who wants to monitor the Darknet — law enforcement must rely on infiltration tactics to find criminals who operate on Tor. An officer could buy or sell drugs undercover, just as one would on the street, but this is more likely to turn up small fish. Agencies can plant downloadable content that contains tracking tools, but Tor vociferously cautions against this. Or — as the FBI did with Ulbricht — they can wait for the kingpin to make a series of small mistakes, accidentally leaving digital breadcrumbs leading to his or her identity. None of these methods, however, are as efficient as law enforcement would likely prefer in weeding out the worst criminals among Tor’s estimated 2 million daily users worldwide. For now, the best way for a Darknet vendor or operator to stay active is to remain a low priority. No child porn. No bombs. No terrorism stuff.
At present, Darknet markets follow a loose self-regulation system whereby the truly dangerous, abusive, or taboo products are prohibited. Vendors agree to terms and conditions that explicitly ban child porn, services related to terrorism or murder, prostitution, Ponzi schemes, and lotteries, to name a few. In April 2014, a Darknet search engine called Grams appeared, which accordingly does not feature such material. Within these loose guidelines, however, the responsibility falls solely on users to evaluate the credibility of those with whom they communicate or transact. Scams, or bad product — such as 3,000 bad ecstasy pills that showed up on Agora Market — are vetted in the forums of each. (In that instance, the person in possession of the bad pills posted photographic proof of flushing them down the toilet.) However precariously, the system works.
As for what the future holds for the Deep Web, our best hints might lie in Tor’s new and planned slate of products. Among others, these include protocols that seem to trade anonymity for improved access: Tor2Web, which allows non-anonymous users to browse Tor hidden services; and Orbot, a Tor browser for Google’s Android platform. Tor has also created Obsfsproxy, a new “pluggable” proxy specifically designed to evade censors. But truest to its original purpose, Tor has also built Tor Cloud, which makes it far easier for everyday users to set up the relays that fortify the anonymity of people who need it, perhaps into perpetuity.
Very interesting piece.