A Modern Plan for Spam
At Handler, we LOVE big bets, especially when they can improve the daily lives of billions around the world. And we believe our lives could finally be cured from spam emails.
In order to achieve what most people consider to be impossible, a spam-free email experience, we’re taking a different and unique approach to anti-spam and building something that’s never been done before.
You don’t need to study or understand much of the economics of spam to fight it. The sole purpose of a spam email is to generate a sale. However, to understand how we are going to eradicate spam, you do need to know what has been our email system’s Achilles heel and why we haven’t been able to get rid of it in the past 3 decades.
So before lifting the curtain on what we do, here’s a brief summary on how spam works and how spammers exploits the weakest spot in our current email infrastructure and network.
Part 1. About Spam
I define spam email as unsolicited bulk emails, wether sent by/for individuals or organizations, including non-profits.
Commercial spam software started taking off in the mid-1990’s (remember Floodgate?).
Probably because the trust level on the Arpanet was relatively high at the time, email protocols didn’t have any real user authentication system and mail relay servers designed in the 1980’s didn’t require outgoing email authentification. Anyone could send emails anonymously.
In response to spam, authentification protocols were developed to prevent sending emails anonymously, and domain authentification, as well as IP/DNS checks, were put in place to prevent domain spoofing.
After authentification, in order to detect and reject potential spam emails from being delivered, or to automatically redirect them to a user’s junk-mail folder, 3 main techniques were developed and are still used today by all email service providers (ESPs).
1. Filtering using machine learning: it consists of applying a set of rules and labels defined by a human, while a computer algorithm learns to recognize and predict potential email spam;
2. Crowdsourcing: every time you press the “mark as spam” button to move emails from your inbox to your junk-mail folder, your ESP collects data to teach and improve its machine-learning filtering system;
3. IP blacklisting: this is the most radical way to block spam, by blacklisting an email server from which email spams are sent.
IP blacklisting was very effective, for a while, until spammers could easily use their own servers and generate new computer IP addresses for themselves. Also, blacklisting become an issue when a single user is on the shared server of a larger organizations (schools, businesses, small ISPs), because when that server is blacklisted, all users are prevented from sending (legitimate) emails.
Machine learning techniques (bayesian filters) have proven to generate fewer false positive (not blocking as many legitimate emails). But spammers always seem to be a step ahead and manage to avoid filters by throwing text-based classifiers off track with their own techniques on a regular basis.
Unfortunately, spam continues to work, both financially and technically.
SPAM IS HUGE
Spam is astronomically huge. The number of spam emails sent in the last 48 hours is greater than the stars in the Milky Way. Every day, near 100 billion emails are sent worldwide and 88% is pure spam.
While spam is very intrusive, it’s also very expensive. It costs American people and companies between $18 billion and $26 billion, every year.
A direct cost of spam is, for instance, when american companies spend $6 billion annually for anti-spam technology and prevention services. There is also the cost of server hardware, which requires 5 times more storage to store their emails than if spam didn’t exist.
The largest email security and anti-spam service provider, Symantec, had $6.5 billion in annual revenue in its latest fiscal year. That also includes its revenues in network security and other bundled offers.
A diffuse cost of spam is, for instance, the time spent dealing with spam emails that got through to your inbox (sorting them, and perhaps reading them) and dealing with false positives and recovering these legitimate emails that went to your spam folders by mistake.
On the other side, spammers and spam-advertised merchants worldwide barely generate $200 million a year in gross revenue. Crazy ratio, right.
So spam is a very expensive and a huge issue. This cat-and-mouse game has been going on for the past 3 decades. So why haven’t we been able to stop the spam pandemic?
THE ACHILLES HEEL
Email addresses aren’t tied to, nor serve, any kind of user authentication purpose. Nevertheless, up to this day, that’s the only thing anyone needs to send an email: a valid email address.
Emails addresses have never been a reliable source of user authentication.
So while ISPs loose playing defense in reaction to billions of spam emails hitting the inbox of their users, spammers hide behind numerous and temporary email accounts and addresses (easily created on temporary IP addresses).
Part 2. Handler
I started working around the basic idea of improving email for people with a new kind of email client — not by making the email interface look better, but rather improving how email actually works, at its core.
Unfortunately the foundations of the legacy email infrastructure and network aren’t giving room to anyone to improve anything about it. Also, I couldn’t reasonably think of using anything from it because nearly 90% of the legacy email system is infected by spam emails.
So while designing and building the foundations of a new kind of email client, I also had to rethink and pioneer a more modern and safer email infrastructure and network. Today, we are calling it Handler.
Can’t Change the Rules, Change the Game.
In the Handler email system, since we couldn’t use traditional email addresses tied to the legacy email system, one of our first objectives was to define another form of user “address” that emails would use to reach each other’s inbox.
This was the perfect time to also plan ahead an offense strategy against future spammers by taking a different approach than what has always been done and never really worked.
At the risk of going against conventional wisdom, we believe that it would be more efficient to prioritize on assessing and controlling the risk of spammers getting into the Handler’s email network in the first place (call it whitelisting if you wish), rather than having to solely focus on containing and reducing potential spam emails in users’ inbox down the road.
THE POWER OF AUTHENTICITY
We wanted our new user address format to also be linked to an authentic user identification system, so we could:
- Make sure users are actual humans, as silly as it may sound. Since the early 2000’s, spam emails are no longer directly sent by humans. They are sent using spamming botnets (ever heard of “Rustock”?);
- Predict more accurately levels of risk of potential spam activity, on several orders of magnitude thanks online public data and social graphs;
- Guarantee the validity of sender & recipient’s address, and have a 100% message delivery rate.
Our team had to entirely rethink emailing authenticity.
Authentic user identification doesn’t necessarily mean using someone’s real and legal name. It means that people are identifying who you are and validate your authenticity. So, if online people are used to address you by a certain name or a username, that’s fine. After all, especially on the Internet, your name doesn’t reflect your authenticity — your activity does.
I believe big time in Twitter, specifically as a new communication protocol. I couldn’t think of using a better format for user & email addresses than the Twitter @username.
We integrated Twitter’s ubiquitous @handle system at the heart of Handler because it can reflect people and emails’ origins in a very authentic way, either when associated with people’s real names or used anonymously. It also bring unprecedented legitimacy, accuracy, and trustworthiness to people’s daily email experience.
We believe that leveraging the power of Twitter in building people’s new email protocol will forever modernize and enhance how the world uses email communications in the delivery of information and ideas, without ever compromising anyone’s privacy.
To be continued…