Multi-Branch CodePipeline Strategy with Event-Driven Architecture and AWS CDK
In AWS’s Multi-branch CodePipeline strategy with an event-driven architecture blog, CloudWatch events are used to trigger a Lambda function that provisions pipelines from a pre-defined Cloudformation template. This enables the use of a GitFlow branching strategy, where developers can create feature branches on a repository and their respective pipelines will be automatically created.
There are further discussions and examples of this approach in the Serverless CICD for the Enterprise quick start found here. This blog, however, describes an evolution of this approach. It looks at implementing the strategy with Typescript and the AWS CDK, and replacing the Lambda and Cloudformation stages with Codebuild and AWS CDK commands.
Related read: Why You Must Modernise Your Existing Applications for Greater Innovation and Agility
Step 1. Create a repository and new branch rule
Firstly, create a Codecommit repository and add a rule for new branches.
Step 2. Create a CDK Codebuild project
Next, create a target Codebuild project for the rule attached to the repository above. The environment variables referenced ($REPO, $BRANCH) are all provided from the Cloudwatch rule payload, which will be set up later on.
Note that the Buildspec also contains a reference to the git-credential-helper, which provides Codebuild with a way to clone our repository at build time.
To complete this Codebuild step, a couple of extra policy statements are required for the Codebuild role.
Here we add policy statements to pull down the code from the new repository, and we also allow the Codebuild role to assume the CDK deploy and publishing roles. These roles are created as part of the CDK bootstrap process.
Step 3. Add the Codebuild target to the rule
With the repository, branch rule, and target Codebuild project created, we can now wire them together. We do this by adding the Codebuild project to the rule as a target.
The key takeaway here is how the rule becomes dynamic by using Cloudwatch event data as a payload to the target CDK Codebuild project. The variables specified in the environmentVariablesOverride are accessible by the Codebuild project, and these are used to clone and checkout the relevant repository and branch, prior to the cdk deploy command.
Step 4. Deploy
Deploy the project into your AWS account with cdk deploy. Once deployed, push your CDK pipeline project into the new repository. Create a branch within the repository, and check everything is wired up correctly:
- The new branch will cause a Cloudwatch rule/event to be triggered.
- In CodeBuild you should see the dynamic pipeline project start, check the environment variables to make sure the right repo and branch are there.
- The CodeBuild project will clone your repository, and check out the branch you just created. It will then attempt to deploy the project with
cdk deploy. - Once the CodeBuild project completes, check CodePipeline to make sure your pipeline is there and running. Its source should be the new branch you created.
Dynamic pipelines
A powerful feature of this approach is the use of context arguments for the cdk deploy command. Context arguments can be provided to your CDK application by using -c, for example -c branch=$BRANCH. This provides a lot of flexibility when deploying your CDK pipeline projects.
The pipeline source code can read the context variable and generate different templates depending on its value. For example:
- If the context argument for branch starts with ‘release’, synth a release pipeline class that roles out to multiple accounts with manual approval steps.
- In all other cases, assume the branch is a feature and use a feature pipeline that only deploys to a development account.
Key pointers
Note that in the dynamic pipeline application example above we use the branch name to set the stackName property. This is essential if you want to have multiple instances of your pipeline in a single account. This property overrides the default stack name, and without this, your Codebuild cdk deploy step will just update the same stack every time it runs, regardless of which branch triggers it.
Also, note that you may get clashes if you are naming your resources and multiple stacks are created in the same account. I removed all references to optional naming properties for my resources, and let the CDK framework allocate them with unique identifiers. On occasions where a name has to be supplied (alerts for example), an option is to reuse the context branch variable as a prefix.
Wrap up
In this blog we looked at the automated rollouts of CDK pipeline applications, avoiding the need for manual cdk deploy commands for new branches. We also looked at using the event environment variables to provide dynamic Codebuild and pipeline applications.
