Cross-Region Import/Export for CloudFormation

While we wait for CloudFormation to become a global service, here’s a little help…

This article is basically a prettier version of the README.md file that you can find in the cfn-cross-region-export Github project.

The problem

When you’re building a multi-region infrastructure using CloudFormation, you’re often faced with the problem of linking resources from a region to another. For example, if you’ve created a Route53 hosted zone for your main domain using a stack in the us-east-1 region but you want to create a DNS record from a stack in the ca-central-1 region, you’ll need to have access to the HostedZoneId. Usually, if both stacks were in the same region you could do a simple Fn::ImportValue but this isn’t going to work this time since that function does not support cross-region referencing. As a workaround, you could decide to use a CloudFormation parameter but this limits the automation that can be done as it requires a manual intervention.

The solution

I’ve been thinking about a way to solve this problem for quite some time and I finally came-up with a solution that I’ve open sourced on GitHub. In a nutshell, it shares the same features as CloudFormation’s Fn::ImportValue intrinsic function, but allows values to be imported from other regions of the same account.

The implementation

The project is divided in 2 parts; the Exporter and the Importer. Only one Exporter stack is needed per region you want outputs to be imported from. The Importer stack on the other hand, need to be instantiated for each
region you want to import outputs from.

Normally, CloudFormation keeps track of which stacks have imported an exported output. This is mainly for preventing an exported output to be deleted while it’s being used by another stack. In order to still benefit from this feature in a cross-region fashion, a stack on the Exporter side is automatically updated to mimic the imports done by other regions.

Here’s an example use-case: Let’s say you are creating some resources in the ca-central-1 region and you need to import values from the us-east-1 and eu-west-1 regions. You’ll need to first provision the Exporter stack in both us-east-1 and eu-west-1 region. You’ll then have to provision 2 Importer stacks in the ca-central-1 region, each targeting a specific region.

Usage

Resources: 
  Importer:
    Type: Custom::CrossRegionImporter
    Properties:
      ServiceToken: !ImportValue 'us-east-1:CrossRegionImporterServiceToken'
      Exports:
        Xyz: 'xyz-export-name'
      
  TestImport:
    Type: AWS::SSM::Parameter
    Properties:
      Type: String
      Value: !GetAtt Importer.Xyz

Installation

The deployment scripts were developed on MacOS, but should work with anything Unix-like. Just make sure you have Python3 and the aws-cli installed, then clone the project and deploy both the Exporter and the Importer as follows:

Exporter

Start by deploying the Exporter

export AWS_DEFAULT_REGION=<EXPORTER_REGION>
make deploy-exporter SENTRY_DSN=https://...@sentry.io/...

Importer

You can find the CROSS_STACK_REF_TABLE_ARN in the output section of the Exporter stack we’ve just deployed.

export AWS_DEFAULT_REGION=<IMPORTER_REGION>
make deploy-importer CROSS_STACK_REF_TABLE_ARN=...

Development

Exporter

Create a DynamoDB table. The python script for the Exporter can be ran locally like so:

export SENTRY_DSN=<A SENTRY DSN>
export GENERATED_STACK_NAME=’dev-ImportsReplication’
export STACK_OUTPUTS_DIGEST_SSM_PARAMETER_NAME: ‘dev-ImportsReplication-OutputsDigest’
export CROSS_STACK_REF_TABLE_NAME=<THE DYNAMODB TABLE NAME>
python3 exporter/lambda/cross_region_import_replication.py

Just make sure you have these permissions attached to your IAM user (or role):

dynamodb:Scan
cloudformation:CreateStack
cloudformation:UpdateStack
ssm:PutParameter
ssm:DeleteParameter

Importer

Since the script importer/lambda/cross_region_importer.py is expecting to be called in the context of a CloudFormation custom resource, I suggest to test your modifications using trials and errors; that means that you edit the script and then deploy it using the method described in the Installation section. You can leverage CloudWatch to help you with the debugging.

TODO (PRs welcome)

• Support cross-account imports (using assume-role it should be fairly easy to do)
• Don’t rely on polling for the ImportsReplication stack.
• Make the SentryDsn parameter optional

I hope this project helps you as much as it helped us maintain our global infrastructure. If you have any questions or need troubleshooting setting up the stacks, just let me know in the comments and I’ll do my best to answer it.

Special thanks to Etienne Talbot, Maxime Leblanc and Simon-Pierre Gingras for the corrections and thanks to Caroline Maltais for the illustration.