Verify domains for SES using CloudFormation

One of the goals of using infrastructure as code is to automate as much as possible the provisioning of your infrastructure. Thankfully, AWS provides us with its CloudFormation tool, which is aimed at just that. One area of the cloud infrastructure that can get particularly messy is DNS configuration (provided by the AWS Route53 service). If you want to (and you should!) manage your Route53 Hosted Zones using CloudFormation, chances are you will have to manage your SES domains through CloudFormation.

In order to send emails using Amazon SES, you need to verify your domain to confirm that you are the domain owner and to prevent others from using it.

This is done by adding a TXT record to your domains’s DNS server. Amazon SES will provide you the name and value for this TXT record.

If you’re using Route53 as your DNS provider, then you’re in for a treat! The SES console will offer you to add the TXT record to your Route53 hosted zone for you, by clicking the blue button:

You must resist the temptation to click that button! Do not forget the 1st commandment of Infrastructure as Code:

Thou shalt not click the dreaded blue button

What you don’t want, is to create those txt records by hand

Using CloudFormation Custom Resources, you can wire a Lambda function to add those TXT records automatically. Our system will look like this:

First, our Lambda function will acquire the verification codes using the SES API. Once our Lambda has these verification codes, it will modify our hosted zone records in Route53.

The Lambda function

First, let’s have a look at the Lambda function’s handler:

def _lambda_handler(event, context):
    print "Received event: "
    print event
    resource_type = event['ResourceType']
    request_type = event['RequestType']
    resource_properties = event['ResourceProperties']
    hosted_zone_id = resource_properties['HostedZoneId']
    physical_resource_id = event.get('PhysicalResourceId', unicode(uuid.uuid4()))
    try:
        if resource_type == "Custom::AmazonSesVerificationRecords":
            if request_type == 'Create':
                verify_ses(hosted_zone_id=hosted_zone_id, action='UPSERT')
            elif request_type == 'Delete':
                verify_ses(hosted_zone_id=hosted_zone_id, action='DELETE')
            elif request_type == 'Update':
                old_hosted_zone_id = event['OldResourceProperties']['HostedZoneId']
                verify_ses(hosted_zone_id=old_hosted_zone_id, action='DELETE')
                verify_ses(hosted_zone_id=hosted_zone_id, action='UPSERT')
            else:
                print 'Request type is {request_type}, doing nothing.'.format(request_type=request_type)
            response_data = {}
        else:
            raise ValueError("Unexpected resource_type: {resource_type}".format(resource_type=resource_type))
    except Exception:
        send(
            event,
            context,
            responseStatus=FAILED if request_type != 'Delete' else SUCCESS,
            # Do not fail on delete to avoid rollback failure
            responseData=None,
            physicalResourceId=physical_resource_id,
        )
        raise  # this statement is important so the exception (along with the original traceback) is logged to Cloudwatch
    else:
        send(
            event,
            context,
            responseStatus=SUCCESS,
            responseData=response_data,
            physicalResourceId=physical_resource_id,
        )

So the basic logic is simple: if our custom resource is created, we’ll create the records in the hosted zone. If it’s deleted, then we’ll delete the records in the hosted zone. If our custom resource is updated, then we’ll need to delete records from the old hosted zone and add records to the new one. Once this is all done, we signal CloudFormation (using our custom send() function, or you could use the cfn-signal helper) that our custom resource is completed.

Now, let’s have a deeper look into the verify_ses() function:

def verify_ses(hosted_zone_id, action):
    ses = boto3.client('ses')
    print "Retrieving Hosted Zone name"
    hosted_zone_name = _get_hosted_zone_name(hosted_zone_id=hosted_zone_id)
    print 'Hosted zone name: {hosted_zone_name}'.format(hosted_zone_name=hosted_zone_name)
    domain = hosted_zone_name.rstrip('.')
    verification_token = ses.verify_domain_identity(
        Domain=domain
    )['VerificationToken']
    dkim_tokens = ses.verify_domain_dkim(
        Domain=domain
    )['DkimTokens']
    print 'Changing resource record sets'
    changes = [
        {
            'Action': action,
            'ResourceRecordSet': {
                'Name': "_amazonses.{hosted_zone_name}".format(hosted_zone_name=hosted_zone_name),
                'Type': 'TXT',
                'TTL': 1800,
                'ResourceRecords': [
                    {
                        'Value': '"{verification_token}"'.format(verification_token=verification_token)
                    }
                ]
            }
        }
    ]
    for dkim_token in dkim_tokens:
        change = {
            'Action': action,
            'ResourceRecordSet': {
                'Name': "{dkim_token}._domainkey.{hosted_zone_name}".format(
                    dkim_token=dkim_token,
                    hosted_zone_name=hosted_zone_name
                ),
                'Type': 'CNAME',
                'TTL': 1800,
                'ResourceRecords': [
                    {
                        'Value': "{dkim_token}.dkim.amazonses.com".format(dkim_token=dkim_token)
                    }
                ]
            }
        }
        changes.append(change)
    boto3.client('route53').change_resource_record_sets(
        ChangeBatch={
            'Changes': changes
        },
        HostedZoneId=hosted_zone_id
    )

Using a helper function _get_hosted_zone_name() (detailed below), we get the name of the hosted zone. Next, using the SES api, we retrieve both the verification and DKIM tokens for our domain. Finally, we leverage the Route53 api to update the record sets on our hosted zone.

Here’s how you can retrieve the name of a given hosted zone. Nothing too fancy:

def _get_hosted_zone_name(hosted_zone_id):
    route53 = boto3.client('route53')
    route53_resp = route53.get_hosted_zone(
        Id=hosted_zone_id
    )
    return route53_resp['HostedZone']['Name']

Finally, here’s the code of the send() function. This is boilerplate code that we carry around our Lambda functions to signal CloudFormation of the outcome of the Lambda's execution:

def send(event, context, responseStatus, responseData, physicalResourceId):
    responseUrl = event['ResponseURL']
    print responseUrl
    responseBody = {}
    responseBody['Status'] = responseStatus
    responseBody['Reason'] = 'See the details in CloudWatch Log Stream: ' + context.log_stream_name
    responseBody['PhysicalResourceId'] = physicalResourceId
    responseBody['StackId'] = event['StackId']
    responseBody['RequestId'] = event['RequestId']
    responseBody['LogicalResourceId'] = event['LogicalResourceId']
    responseBody['Data'] = responseData
    json_responseBody = json.dumps(responseBody)
    print "Response body:\n" + json_responseBody
    headers = {
        'content-type': '',
        'content-length': str(len(json_responseBody))
    }
    try:
        response = requests.put(responseUrl,
                                data=json_responseBody,
                                headers=headers)
        print "Status code: " + response.reason
    except Exception as e:
        print "send(..) failed executing requests.put(..): " + str(e)

The CloudFormation template

First, you’ll need a Lambda function:

AmazonSesVerificationRecordsLambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Description: This function manages the verification and DKIM records for SES
      Code:
        S3Bucket: my-bucket-name
        S3Key: ses_route53_verification.zip
      Handler: ses_route53_verification.lambda_handler
      Role:
        Fn::GetAtt:
        - AmazonSesVerificationRecordsRole
        - Arn
      Runtime: python2.7
      Timeout: 30

The timeout can be left to 30 seconds, as our Lambda is short to execute. You will also need to store your Lambda’s zip in a bucket.

Next, we’ll need an IAM role for our Lambda:

AmazonSesVerificationRecordsRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Sid: 'AllowTheLambdaFunctionToAssumeThisRole'
          Effect: Allow
          Principal:
            Service: lambda.amazonaws.com
          Action: sts:AssumeRole
      Path: "/"
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
      - PolicyName: Route53Access
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - route53:GetHostedZone
            - route53:ChangeResourceRecordSets
            Resource:
            - !Sub arn:aws:route53:::hostedzone/${MyHostedZone}
      - PolicyName: SesAccess
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - ses:VerifyDomainDkim
            - ses:VerifyDomainIdentity
            Resource: "*"

We give permissions to our Lambda to use CloudWatch logs (using the built-in AWSLambdaBasicExecutionRolepolicy), change record sets on our hosted zone and generate the verification values.

Finally, here’s our custom resource:

SesVerificationRecords:
    Type: Custom::AmazonSesVerificationRecords
    Properties:
      ServiceToken:
        Fn::GetAtt:
        - AmazonSesVerificationRecordsLambdaFunction
        - Arn
      HostedZoneId:
        Ref: MyHostedZone

Our custom resource will invoke our Lambda and pass in the ID of the target hosted zone that needs to be modified.