Blacklisting Emails with Amazon SES and Lambda

Why you should care about bounce rates.

Published in

Divvy Engineering

4 min readNov 7, 2019

“High bounce rates are often used by entities such as ISPs, mailbox providers, and anti-spam organizations as indicators that senders are engaging in low-quality email-sending practices and their email should be blocked or sent to the spam folder.” [1]

A bounce rate over 5% puts your account under review and a bounce rate over 10% can result in a suspension of your account. 😱

Overview

At Divvy (https://getdivvy.com/) we run an smtp proxy that validates emails, by checking an email address blacklist, prior to calling Amazon SES. This article describes how we blacklist email addresses.

blog.golang.org/go-brand and synth.agency/lab/flat-osx/

In order to blacklist emails, to prevent bounces, we are going to use the following services:

SNS Topic
Go Lambda Function
Database/Persistent Storage

I will describe how to create the SNS Topic, SES Notification, and the Lambda function with terraform at the end of this article, but for now, let’s focus on the golang lambda function.

Email Blacklisting Lambda

Our blacklisting lambda function will subscribe to the bounce notification SNS topic, meaning it will be the recipient of bounce events. Start by defining a BounceMessage struct.

main.go — defining a BounceMessage

This allows us to conditionally handle different bounce types and access the bounced email address. The BounceType will be either Permanent or Transient. You can read more about the different bounce types here. We are only interested in blacklisting permanent bounces. Transient bounces may live to bounce another day.

Before we write the logic to blacklist an email, let’s stub our event handler and write a test. The event handler will return an error on failure and nil on success.

main.go — lambda entrypoint

main_test.go — simulating a SNSEvent

We are now ready to unmarshal these JSON strings, check the bounce type, and conditionally blacklist an email. Which is where our persistent storage comes into play.

Database

Here is where our solutions will likely differ given the diverse set of database options. I think DynamoDB would be a great choice here (AWS Dynamo SDK), however, I used AWS RDS for legacy support reasons. The following snippet will be a good starting point if you do the same:

data_provider.go — https://github.com/go-sql-driver/mysql/

If you want to go the extra mile consider adding some unit tests and using go-sqlmock.

Completing Our Lambda

Once you’ve implemented a data provider we can return to our main.go event handler and add the logic to blacklist emails.

main.go — SNSEvent Handler

We iterate over each event record, ignoring the index, and unmarshal the string resulting in a BounceMessage . We then check the bounce type, and if it's permanent, insert the email into our database. And that’s it. It’s pretty straightforward.

Prior to sending an email using SES you can now programmatically check your blacklist (be sure to index on the email address).

Regardless of what persistent storage provider you use, I recommend keeping your main package lightweight by separating your data provider code into its own package [2].

AWS Resources with Terraform

Now that we have the src for our Lambda function, we need to deploy it and fill in the other missing components.

The AWS SES bounce action rejects the email by returning a bounce response to the sender, and optionally notifies you through SNS [3]. Let’s start by defining our terraform code for the SNS topic.

We will also need a IAM role and policy attachment to enable Cloudwatch logs, both of which we will define with terraform.

And finally let’s define our SNS topic subscription, the lambda function itself, and lambda permissions.

In case you are unfamiliar with terraform and lambda, the filename is the path to your zip containing your go executable, and handler will be the name of the executable itself.

# for example
BINARY_NAME=email_blacklistCGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o $(BINARY_NAME) -v <path_to_main>zip -r $(BINARY_NAME).zip $(BINARY_NAME)

You may choose to manage your SES domain separately because it has dependencies external to our blacklisting functionality. However, we can still configure the notification itself with the following terraform code.

AWS Resources without Terraform

Optionally, if you choose not to use terraform you can complete the steps above with the aws-cli or AWS dashboard. You will need to create the same AWS resources including:

SES identity notification topic
SNS Topic
Lambda Function which should use the role entitled AWSLambdaBasicExecutionRole
Lambda SNS trigger

For example, you can create an SNS topic with the following command using aws-cli

aws sns create-topic --name <sns-topic-name> \
--attributes '
{ 
    "DisplayName" : "<sns-topic-display-name>" 
}' \
--tags '
[
    { 
        "Key" : "description",
        "Value" : "notification topic for email bounces" 
    }
]'

Once your src code has been zipped, and AWS resources deployed, you should start to see Lambda triggered by bounce events. Check your Cloudwatch logs to validate this behavior.

I hope this article helps you understand more about SES email bounces, SNS, Lambda and Go. Its time for me make like an email and bounce.