Demystifying the Darknet: Part I
Getting Adele’s Message to Eddie Murphy Anonymously
The Darknet is one of those topics that when it’s mentioned over cocktails people generally nod while quietly hoping that the point being made doesn’t rely on actually understanding it. While the concept of the Darknet is becoming mainstream (e.g., see these TED talks), it’s still not there yet. As a consequence, finding a satisfying explanation of the Darknet is nontrivial. This will change over time, but it won’t happen on its own. It’s up to those of us that understand the Darknet to make its concepts accessible to others. What follows is my modest attempt.
When using the Darknet from your laptop, there is some networked machine that is your ultimate destination — that you’re ultimately wanting to communicate with (e.g., a machine hosting a web server). This destination machine may have a presence on the Darknet or the non-Darknet — commonly referred to as the “Clearnet.” This post is only the first installment of a two-part post. In Part I, we will deal with the case when the destination machine is on the Clearnet. In Part II, we will deal with the case when the destination machine is on the Darknet.
Deconstructing the Term Darknet
“What’s in a name?” ~Juliet in William Shakespeare’s “Romeo & Juliet”
Well, let’s deconstruct “Darknet” and find out. Darknet is comprised of two words: namely, “Dark” and “net.” Let’s examine each word in turn.
“Dark” Refers to Anonymity
Given that the “Dark” in Darknet corresponds to anonymity, what does it mean to be anonymous? Being anonymous means knowing at most one of the following with respect to some situation: who you are or what you’re doing. If you robbed a bank (situation) with full body gear, gloves, mask, and voice changer, nobody would know who you were; however, people would know what this “stranger” did. Consequently, in this situation, you’d be anonymous. If your sister, who knows nothing of your robbery is asked about the crime, she would certainly know who you are but know nothing of what you did. Consequently, in this situation, you’d be anonymous. Now, if on the day of the robbery, you chose not to use your mask and your sister was present and witnessed the robbery, you could kiss your anonymity goodbye. Your sister would not only know who you are, but also what you did.
“Net” Refers to Network — an Overlay Network
This is going to sound very “Captain Obvious-ish,” but bear with me. An overlay network is overlaid atop of another network. So, what does it mean for one network to be overlaid atop of another? It means the overlaying network leverages the services of the underlying network. An overlay network is adhering to the well-known principle of reusing capabilities. A beautiful aspect of an overlay network is that it allows for a new overlay network to innovate where it has innovations and to delegate to an underlying network where it has none. If I have clever startup idea to get Web content to end users more quickly through an overlay network called a Content Delivery Network, do I really need to re-implement the whole TCP/IP stack??? If I do, that’s a total non-starter. Thankfully, the Internet we’ve all come to know and love was built to enable such reuse via its modular design.
Defining the Darknet
Substituting words appropriately, we can deduce that the Darknet is nothing more than an anonymous overlay network.
Common Misconceptions Worth Dislodging
In chatting with people of all technical levels, I’ve found common misconceptions that make understanding the Darknet more troublesome than it needs to be. What follows are the ones that came up repeatedly.
Misconception #1: The Internet is the Web
The tricky thing here is that most of the time these terms are interchangeable in the common ways people use them. “Hey honey, could you check the Web to see what time the game comes on?” Or, equivalently, “Hey honey, could you check the Internet to see what time the game comes on?” Despite this apparent interchangeability, the Internet is not the Web. What makes this difficult to see is the Web’s dominance on the Internet — it is the killer app. Any really useful Layer 7 (from the OSI model) application usually gets a Web interface at some point. Email is a great example. Because so many of us access our Gmail accounts through a Chrome browser, we can easily forget that there are desktop email clients that are not Web browsers (e.g., Mozilla Thunderbird) but still connect to the Internet. How can this be? Well, in the TCP/IP Model (the model any machine on the Internet must implement), one of the responsibilities of the Transport Layer is to connect to remote hosts. So whether we’re talking about the Web (HTTP/S) or email (SMTP, POP, IMAP) or some other application at Layer 7, if the application connects to the Internet, it relies on the Transport Layer (and each layer beneath it).
Misconception #2: Darknet, Dark Web, Deep Web, and Black Market are All the Same Thing
Let’s deal with the most egregious term that’s sometimes used as a synonym for the others: Black market. The Black market is a marketplace where people buy and sell illegal goods or services. As long as there have been laws related to commerce, there have been people circumventing them. So while it’s true that black markets exist on the Darknet, black markets existed well before the Darknet existed. Moreover, the Darknet houses plenty of other things beyond Black markets: email, chat rooms, personal websites etc.
The Dark Web is on the Darknet in the same way the Web is on the Internet. We introduced the term Clearnet earlier to refer to the non-Darknet. And since the Clearnet exists, there’s also a Clear Web. And as you might expect, the Clear Web sits atop the Clearnet. It’s the Clear Web that most of us are referring to when we say the “Web.” So, a more specific statement we can make is that the Dark Web is on the Darknet in the same way the Clear Web is on the Clearnet. The Dark Web is nothing more than HTTP/S on the Darknet. (We’ll focus more on the Dark Web in Part II.)
The Deep Web is quite different from the Darknet. One of the giveaways is the term “Web” in its name. The Deep Web is only concerned with the Web (only one of many different types of layer 7 protocols you find on the Darknet).The Deep Web refers to the Web pages that are not picked up by Web crawlers. As a consequence, you cannot visit Web pages on the Deep Web via a search engine. Generally, the types of Web pages that Web crawlers do not crawl are Web pages that are behind a paywall or login page. Since Web crawlers on the Clearnet do not crawl the Dark Web, the Dark Web is also part (a proper subset) of the Deep Web.
Misconception #3: The Darknet is TOR
TOR stands for “The Onion Router.” More importantly though, TOR is an example of a Darknet. While it’s arguably the most popular example, at the end of the day, it’s only an example of a Darknet. This doesn’t stop folks from referring to TOR as “The Darknet.” This reference is similar to a habit some in the San Francisco Bay Area have. If you’re new to Bay Area, and you ask someone “so, what’d you do this weekend?” it’s fairly common to hear a response that has the following phrase in it “…I was in The City…” To someone new to the area, this response will be quite confusing as you’ll recall there being many cities in the Bay Area. But, that’s where you’d be wrong. For some in the Bay, the only city that “matters” is San Francisco — so there’s actually zero ambiguity. And in the same way some think San Francisco is the only city that matters, some believe that TOR is the only Darknet that matters. And in case you’re wondering, there are other Darknets. For example, one that occasionally comes up is the Invisible Internet Project (I2P).
Focusing on TOR
Since most people are referring to TOR when they say “Darknet”, we’ll now get into how TOR works. The hope is that this treatment will give a good sense of how the most popular Darknet works for a given use case: sending messages to the the Clearnet. Often, when this aspect of TOR is being briefed, the following Electronic Frontier Foundation (EFF) picture is shown:
While this image succeeds in providing a high-level representation of what’s happening on TOR to reach a machine on the Clearnet (Bob), in my experience, people still feel uncomfortable with what’s going on. The discomfort can be summed as follows: “what’s so Darknet about this?” Why do you get anonymity with this? What is the overlay network doing? Where is the overlay network? What is it on top of? How in the world do I get on it?
What follows is designed to address these questions.
Using TOR to Get Adele’s Message to Eddie Murphy
After a long hiatus from doing a stand-up comedy film, Eddie Murphy has finally decided to do it again one last time!
After the success of Eddie Murphy Delirious and Eddie Murphy Raw, people far and wide are eager to see this new stand-up comedy movie. Eddie Murphy knows that people are eager and wants to make sure the film is a major hit. Every little detail matters. He put out the word that he’s looking for someone to do the intro music for his new movie. He’s indicated that interested parties should mail suggestions or demos to his home address.
Adele caught wind of this, and, being a huge fan of Eddie Murphy’s work, absolutely wanted to be chosen to do the intro music for this new film.
However, she doesn’t want him to know that she’s recommending herself. Being a superstar music artist, Adele knows that perception is everything. After much deliberation, she decided that a letter recommending her would be stronger if it comes from somebody else…preferably someone that is noteworthy but more importantly, lovably trustworthy: Drake.
Because Adele is very careful, she enlists the assistance of a couple more helpers. The first two people to come to mind were Beyonce
and, surprisingly, Charlie Murphy.
Since Charlie Murphy is Eddie Murphy’s brother, superficially, it seems that it would be risky to have him involved. Being Eddie Murphy’s brother his loyalty is likely with Eddie Murphy. So, if Charlie Murphy is able to de-anonymize the message he may be able to tell Eddie Murphy who really wrote the letter. However, we will later see that Charlie Murphy will not learn enough to tell Eddie Murphy anything useful.
So, after Adele composes a letter that recommends herself to Eddie Murphy, she not only doesn’t sign the letter, she leaves the return address area blank in the letter and the envelope. She only puts the destination address: Eddie Murphy’s home address.
Now that Adele has determined that Drake will send the letter to Eddie Murphy, she now has to determine who will send the letter to Drake. She decides Charlie Murphy will be the person to do it. This leaves Beyonce as the person to deliver the package to Charlie Murphy. This arrangement means that Adele will need to send her package to Beyonce first.
With the ordering determined, Adele has managed to obtain lockboxes and keys for Beyonce, Charlie Murphy and Drake. The key Adele shares with each of these individuals isn’t shared with anyone else. So, while Beyonce can open her lockbox with the key she shares with Adele, she cannot open up Charlie Murphy’s lockbox or Drake’s lockbox.
To begin the process, Adele puts the envelope into Drake’s lockbox.
She then puts Drake’s lockbox into Charlie Murphy’s lockbox.
She then puts Charlie Murphy’s lockbox into Beyonce’s lockbox.
This leaves Adele with a single package.
Once Beyonce gets the package, she unlocks her package, and sees that it is addressed to Charlie Murphy.
She adds her return address to Charlie Murphy’s package in preparation for her local post office.
Once Charlie Murphy gets the package sent by Beyonce, he unlocks it. He sees that within his package is another package that is addressed to Drake.
After cursing out Drake, he dutifully puts his return address on the package in preparation for the post office (Charlie Murphy curses out Drake because Charlie knows he’ll have to pay more since Drake lives in Canada).
Once Drake receives the package, he unlocks the box and sees the envelope addressed to Eddie Murphy.
Drake promptly puts his return address onto the envelope in preparation for his local post office.
Finally, Eddie Murphy receives the letter at his residence. Although Adele wrote the letter, by all appearances and indications the letter came from Drake. Eddie was impressed that Drake was willing to recommend someone else. Given Drake’s popularity, Eddie thought if Drake would send a letter at all, Drake would be recommending himself. Having considered “Drake’s words,” Eddie Murphy agrees and recalls how Adele totally rocked Skyfall and thought Adele could likely do something similar for his stand-up comedy movie. He responded to Drake via mail saying that he liked the idea and that Adele was his number one choice.
Once Drake receives this letter at his home address, he puts it into his own lockbox, locks it and addresses it to Charlie Murphy. When Charlie Murphy gets the package, he puts the package into his lockbox, locks it and addresses it to Beyonce. Once Beyonce gets the package, she puts it into her lockbox, locks it and addresses it to Adele. Since Adele has all of the required keys, she opens all of the lockboxes and gets Eddie Murphy’s response. Upon reading it, she was elated — so much so that she decided to do a rendition of the Eddie Murphy’s “Party all the time” just for fun.
Mapping the Metaphor to Reality
- Adele = TOR client (if the destination is on the Web, then the TOR Browser is a popular choice as it’s a modified Firefox browser — this is probably the quickest way to jump onto a Darknet)
- Beyonce = TOR Guard Node
- Charlie Murphy = TOR Middle Node
- Drake = TOR Exit Node
- Eddie Murphy = Clearnet destination server (e.g., a Web server like yahoo.com)
- Lockbox (locked) = AES-128 ciphertext
- Key = AES-128 key
- Letter = message (e.g., HTTP Request)
- Envelope/Shipping Labels = Network/Transport Layer information (i.e., source and destination IP & port information)
- Postal Service = Internet
- Path from Alice to Drake = Virtual Circuit
So, was Adele Anonymous?
To answer this question, let’s recall that in order for Adele to not be anonymous, Beyonce, Charlie, Drake, or Eddie would need to know that Adele is sending a letter and that this letter is destined for Eddie Murphy. So, let’s look at what the different participants know. Beyonce knows that Adele is sending a package. She also knows that she’s sending the package to Charlie Murphy. Beyonce knows nothing about Drake or Eddie Murphy. Charlie Murphy knows that Beyonce is sending him a package and he knows that he’s sending that package to Drake. Charlie knows nothing about Adele or Eddie Murphy. Drake knows that the package contains a letter and knows that it’s destined for Eddie Murphy. Drake knows nothing about Beyonce, or Adele. Eddie Murphy thinks the letter is from Drake and only knows about Drake.
By examining what each of these people know, a critical property is revealed: no person knows the ultimate source and the ultimate destination of a message handled. While Beyonce, knows that Adele is sending a package, she doesn’t know that it’s destined for Eddie Murphy. While Drake knows that the letter is ultimately for Eddie Murphy, he doesn’t know Adele sent it. Charlie Murphy only has awareness of Beyonce and Drake. Eddie Murphy only knows Drake was involved. None of these people know both who Adele is and what she is doing (with respect to the situation of communicating with Eddie Murphy). As a result, Adele’s communication to Eddie Murphy is anonymous.*
*A Word of Caution
Reality is more complex than this. A concrete example is attempting to surf the Web anonymously. Let’s say you started looking up instructions on how to construct a bomb and how to obtain the required materials on the Web by sending your traffic through TOR. It’s fair to assume that if your browser isn’t fully patched and configured properly, your browser will be compromised by some government agency. And if this agency has access to your local browser, they can see everything you’re doing on the Web (and maybe more). If you’re interested in more examples of things that may harm your anonymity while on TOR, you may want to consider this “Do Not Do” list.
Why the “O” in TOR?
You now know why TOR refers to “The Onion Router.” Everyone except Adele and Eddie Murphy are considered Onion routers. Beyonce, Charlie Murphy, and Drake are responsible for peeling off layers of a onion for messages destined for the Clearnet (e.g., Adele’s letter to Eddie). The act of peeling off layers is captured by their unlocking (decrypting) their respective boxes. This same group of three is also responsible for adding on layers to an onion for messages destined for the Darknet (e.g., Eddie’s letter to “Drake” that unbeknownst to Eddie is really destined for Adele). The act of adding layers is captured by the group putting packages into lockboxes (encrypting).
So, Where was the Overlay Network and the Underlying Network?
The overlay network corresponds to the set of actions (protocol) that Adele, Beyonce, Charlie Murphy, and Drake followed. All of the putting packages into others or taking packages out of others is all part of the overlay network. The underlying network corresponds to the Postal Service. None of these famous folks are in the business of personally delivering messages. Ergo, they leave that grunt work to the Postal Service. If you read the mapping section, you know that the Postal Service corresponds to the Internet. So, this means that TOR is using the same Internet infrastructure (e.g., routers, ISPs) that we all already use.
What You’re Now Able to Do
If you started out knowing nothing about TOR but followed the above Adele example, you now know enough to start asking and answering your own questions about TOR. Curious to know how Adele picks her cast of helpers? Maybe you’re curious about how exactly they ended up getting their respective keys? Maybe wondering if she really needs three helpers? Whatever your question, you can probably find the answer in the TOR specification or technical paper. And if you can’t find the answer to your question there, you probably have a good research question (but do a literature search first :-).