How I built Heimdall, an open-source personal email guardian.

Like the Asgardian protector, it valiantly guards your inbox.

Fabian Terh

Published in

The Startup

8 min readFeb 1, 2020

I love receiving spam mail! — No one ever.

I use Gmail, which does a fantastic job of weeding out spam and sending them straight into the spam folder. Over time, however, it’s still frustrating watching the spam folder count increase and having to periodically sift through it for false positives.

The types of spam email that irritate me most are not emails claiming of generous Nigerian fortunes or wondrous enhancement drugs, but emails from legitimate companies who I’ve never interacted with before. So how did they get my email? Someone must have sold or disclosed my email address to them and I would really like to know who they are.

But there is no way of knowing exactly who disclosed my email address, and no way of taking action/stopping the email, right?

Well, not quite.

Disposable, single-use email aliases allow you to shield your actual email address, while forwarding all received emails to your personal inbox.

(Link to project.)

Why use Heimdall

To be clear, this idea certainly isn’t novel.

A quick Google search turns up a number of different products and services that do exactly this.

I could easily have opted to use any of those, but here are three reasons why I decided to build Heimdall:

Third-party services are either paid, or offer a free tier with limitations. Depending on which service you use, you may or may not find the limitations acceptable for your use cases.
Using a third-party service means having to trust that company with your emails. Most services are closed-source and proprietary — you can’t ever be sure that your data isn’t stealthily harvested. Even if it’s open-source, you can’t be 100% sure that the service you are using is running on the same codebase.

Even as far as open-source projects go, Heimdall isn’t the first or only one. There are a bunch of those on GitHub. But from what I could tell, those projects are one or more of the following:

They are codebases of actual services (either free or paid) and does not include setup instructions. Heimdall is meant for individual users to deploy and use and contains user-friendly setup instructions.
They have very complicated setup instructions, including server configuration requiring in-depth technical know-how. Heimdall runs on serverless computing, so there is zero server configuration or provisioning.
I actually found a project that is very similar to Heimdall (it runs on AWS Lambda and SES) but contains manual, lengthy setup instructions. Heimdall uses the Serverless framework (not to be confused with small-letter serverless in Point 2 above) so you can deploy with a single command.

But more importantly, I built Heimdall because it’s fun! I’ve always enjoyed working on pet projects, and this is a great opportunity for me to pick up AWS and learn about the Serverless framework.

Infrastructure

I’m using AWS’s Simple Email Service (SES) to send and receive emails, S3 for storage, and Lambda functions for serverless computing. Here’s how it works:

All received emails trigger SES to store the email as a file in a S3 bucket, which triggers a Lambda function. Depending on the email, one of several things could happen:

The email gets forwarded to your personal email address
The email gets forwarded to the original sender (when you reply)
A command is invoked by you (e.g. to generate a new alias)
Nothing happens (when someone emails an invalid/disabled alias)

I chose to use AWS for practical reasons: I’m totally new to cloud computing, and AWS being the most popular cloud computing service means it is easier to find guides and resources online.

Commands

Most email aliasing/forwarding services provide either a web or mobile application to generate, update, or delete aliases. Building a web application — even a simple one — would require significant effort and resources.

Since Heimdall is already in the business of sending and receiving emails, I thought: “Why not let commands be issued through email too?”

Command emails are used to generate, list, update, and delete aliases. For instance, if I want to generate a new alias, I’d email generate@mydomain.com. Apart from the whole laziness thing, I thought this is a really cool and convenient way to interact with the service.

This alias is actually valid and live! Feel free to say hi, but please don’t spam it.

Receiving emails

Forwarding emails that are received on aliases to the user’s personal email address is perhaps the most important function. In building this functionality, there were a few considerations at the forefront:

Minimal, non-destructive modifications

As much as possible, I want to preserve the information contained in the original email — user-visible or otherwise. For instance, if the email was sent by Acme Corporation to me while cc-ing Alice and Bob, it should arrive in my personal inbox with Alice and Bob still in the cc list. Removing them would be a destructive modification to the original email, because I will have lost some of the email’s original information.

Heimdall should operate as transparently as possible, as if it wasn’t even there.

Here, I used my secondary email address to send an email to the alias I created above, while CC-ing my university email address. As you can see, the CC information is preserved.

Stealthy privacy

To do the above, all I have to do is to keep Alice and Bob on the cc list when forwarding the email from the alias to my personal email address, right?

Nope. Doing that means Alice and Bob gets a second copy of the same email. Plus, they would be able to see that the email has been forwarded to my personal email address. So much for privacy.

Therefore, what I need is to forward the email to my personal email address, keep Alice and Bob on the cc list, but not have them know about it.

This is a surprisingly tricky feature to build, and resulted in a day of diving deep down into a rabbit hole on how email works. I wrote an in-depth technical article about that here. It has to do with the magic that is SMTP envelopes and email headers.

Photo by Richard Braggins-Taylor on Unsplash

Replying anonymously

For most use cases, being able to receive emails anonymously and reliably is adequate. But for Heimdall to be feature-complete (at least as a minimum viable product), it should also support replying to emails anonymously.

To implement anonymous replies, I had to figure out a way to include information about the original sender (OS)’s email address in the forwarded email. Obviously, replying directly to OS is out of the question (it exposes your personal email address in the “from” header). Associating an alias with an OS is also wrong, because an alias could potentially receive email from multiple senders.

What I wanted was a way to easily reply to the original sender while maintaining anonymity.

Eventually, I opted to include OS’s email address in the email address to which replies will be sent. The easiest way to do this is to set the “from” header in this forwarded email (from alias to personal email) in such a way as to include information about OS’s email. To accomplish that, I’m using this format: alias+originalSenderEmailAddress@mydomain.com.

For sanity of parsing, I encoded originalSenderEmailAddress in Base64 representation. This prevents multiple “@” symbols from appearing, such as in alias+marketing@company.com@mydomain.com. This is weird as hell and would probably screw up the email address parsing library that I’m using. Base64’s character set doesn’t include the “@” symbol, nor the “+” symbol, which makes it extremely apt for this purpose — it’s guaranteed that there will only be one “+” separator character and one “@” character.

Therefore, as an example, an email sent from marketing@company.com to alias@mydomain.com will be forwarded to my personal inbox as being from: alias+bWFya2V0aW5nQGNvbXBhbnkuY29t@mydomain.com.

You can see this clearly in the screenshot above (the Base64 representation of my secondary email address is contained in the long, blacked-out portion of my alias).

Finally, when Heimdall receives this email (and detects it as originating from my personal email address), it parses the email address to obtain alias and marketing@company.com (after decoding). It repackages the email as being sent from alias@mydomain.com to marketing@company.com, while preserving the email subject and body.

Tests

So far, I’ve described an intricate system of multiple logical components working together. Considering that email is a critical service — in many circumstances, it is the primary channel of communication — I need to be 100% sure that my implementation works correctly.

I built Heimdall with reliability as my utmost priority. In this case, reliability refers to the fact that new features or bug fixes do not accidentally break something else. (Since the service is run on AWS, I have no control over infrastructure reliability.)

In the early stages of development, I set up continuous integration on Travis CI and integrated Code Climate to track code quality and test coverage. Almost all business-logic modules are 100% unit-tested, and many are integration-tested as well. With a comprehensive set of tests covering almost every aspect of functionality, I can be relatively confident in deploying new builds (although there is no such thing as perfect code!)