What Exactly is End-to-End Encryption?
Recently the press has been alight with news that Apple iMessage, WhatsApp, and Facebook Messenger now support something called “end-to-end encryption”.
But what exactly is end-to-end encryption you may ask? Well, my company Inky makes an end-to-end encrypted email/calendar app, so let me take a crack at explaining it in layman’s terms.
First, as you probably know, encryption refers to the use of a cipher to obfuscate a message. The earliest ciphers were letter substitution ciphers; these emerged in antiquity, and basically involve systematically substituting one letter for another throughout a text. The famous Caesar Cipher, for example works by lining up two alphabets in a row and replacing each upper letter with the equivalent lower letter:
ABCDEFGHIJKLMNOPQRSTUVWXYZ
TUVWXYZABCDEFGHIJKLMNOPQRS
So
INKY
becomes
BGDR
Notice how in the bottom row the letters are still in the same order, but they just start at a different letter and then “wrap around.” This “rotation” characterizes the Caeser Cipher and is also used in another common cipher called “Rot-13”. It’s been suggested that perhaps a cipher of this sort was used to obscure the text of the mysterious Voynich Manuscript:
With the advent of practical computing devices — initially mechanical and soon after much faster electronic ones — it became obvious that letter substitution ciphers couldn’t possibly be secure. In the case of a rotation cipher, it’s easy for a computer to just try all the possible rotations of the alphabet; only one will produce the intelligible “plain text” of the original message.
By World War II, mathematicians were using cryptanalysis — a collection of techniques used to break ciphers. You may have heard of the famous case of Alan Turing and Polish mathematicians breaking the Germans’ Enigma code. Ciphers of this era (and today) use keys — secret values used to scramble the exchanged message. To communicate via Enigma, for example, both sides would have to set mechanical wheels on the device to the matching values — this is essentially what cryptographers now call a pre-shared key, because it must have been exchanged ahead of time over another secure channel. One German operative might give another German operative a piece of paper with the key on it when both were physically together at headquarters and could verify each other’s identities and “authenticate” each other. The key might even be on rice paper so that if an operative were captured he could eat the key to prevent the enemy from getting it!
A simple cryptanalytic method that allows one to trivially break substitution ciphers is frequency analysis. Intuitively, the letter e occurs more frequently than any other letter in English text. So if we’re looking at text encrypted with a substitution cipher and we see more q’s than anything else, it’s likely e has been replaced with q. This is actually straightforward enough that Games Magazine has pencil-and-paper puzzles that you can solve using this exact strategy.
(As an aside, one thing that sets the Voynich Manuscript apart from other historical obfuscated documents is that it seems to defy frequency analysis: the distribution of its “letters” doesn’t seem to match that of any known language, suggesting something more subtle is going on.)
These days the ciphers we use are quite well understood mathematically, and are believed to withstand not only sophisticated cryptanalysis but brute force attacks by very fast computers as well. In fact, a single message encrypted with today’s standard AES cipher and a 256-bit key is thought to require a computer marshaling every atom in the universe to compute until well beyond the end of time to break by brute force. Cryptanalysis these days focuses heavily on the implementation of ciphers in program code — finding subtle bugs in the way the programmer coded, say, the AES algorithm, that accidentally leak timing or other information — and that can be cleverly exploited to reduce the amount of brute force work required.
Practically speaking, then, encryption is pretty much a solved problem — so much so that it’s actually built into the hardware of your phone now. When you send a text using iMessage, your iPhone hardware is encrypting the message using essentially unbreakable encryption. And you don’t even notice — awesome!
But what about this “end-to-end” business?
End-to-end is a geeky term that basically means that only the sender and recipient can ever decrypt the messages — i.e., that decryption is only possible on a so-called endpoint device like your phone. Importantly, nobody in the middle can decrypt anything — not Apple, who runs the servers; not Verizon who controls the physical fibre your message is sent over; not spies tapping into an undersea cable; nobody.
But isn’t this the entire point of encryption — preventing parties aside from the intended recipients from reading the encrypted messages? So why would anyone not use end-to-end encryption?
Well, email is typically not end-to-end encrypted unless you’re using Inky or a mail reader supporting something called “Pretty Good Privacy” (PGP)— but why not? There are a few reasons.
The first is that the email standards predate all this concern about security on the internet. When Ray Tomlinson sent the first email back in 1971, nobody was thinking about the barrage of cyberattacks we face today. And, quite possibly, nobody imagined we’d still be using the same email standards in 2016! Building the internet was hard enough on its own; the pioneers who did it didn’t have time to worry about all the future security implications.
The second is that, with the advent of webmail, all your email is stored on your provider’s servers. So if you have a Gmail account, Google is storing your mail on some server(s) somewhere around the world. (Where else would it be?)
Furthermore, webmail clients don’t typically do much heavy lifting; they’re usually pretty simple web pages that just display text and graphics and take input from you. The server is where the hard work like search and filtering happen. (It’s worth noting that for some tasks, like Google search, the server is the only place where search can happen, because the data being searched is too big to fit in your device. But that’s not true of your email.)
So when you ask Gmail to tell you what messages in your inbox match some text, it’s Google’s servers that are doing the searching, not your computer or phone.
But of course this means some piece of code that’s not on one of the endpoints must be able to decrypt the email —the piece of code running on Google’s server! Otherwise, how could it search? (Actually, there is a way it could work without decrypting your mail, but techniques for searching encrypted data are purely academic at this point and not yet practical.)
So for hosted webmail, the servers have your decryption keys as well, rendering the encryption, well, sort of useless: the efficacy of encryption is limited by who has the keys, and in the case of a huge company hosting your email, you can’t possibly know who has the keys.
At the end of the day, end-to-end encryption is what we should all be demanding from our cloud vendors. They’ll need to change their architectures to provide this — to perform searches on our phones and desktops, and right in our browsers like Inky does — but in the long run it’s essential for our privacy and security.
It’s an investment all cloud providers need to make. Now.
