Using CodePipeline to Automate Serverless Application Deployment

AWS offers a world of cloud services that give us the ability to move our workflow into a third-party physically managed infrastructure. In this article, I intend to show how we can adapt and automate the simple use case of continuous deploying a serverless application, such as a REST API with NodeJS, exclusively using AWS services.

Introduction

Let’s say you have an application in, for example, NodeJS — you can be using AWS Lambda and possibly connected to AWS API Gateway. If you haven’t yet automated this process (externally), you probably look like the octopus above, sharing your pre-release efforts into each one of the tasks. At certain point, you ask yourself, how can I bring the most recent release into staging/production rather than manually building or deploying it?

The desired end-state for the pipeline, including two optional deployment environments, Staging and Production.
  • Stage 2 (Build): CodeBuild to do the “build” or “bundle” parts, this is as simple as zipping up any artifacts you have for NodeJS functions. In the end, you will have a file specification, called buildspec.yml, to tell CodeBuild what files make up the final build artifact.
  • Stage 3 (Deploy): CodeDeploy will call CloudFormation to create or update the lambda function with the most recent code. While this process occurs, it will be responsible for versioning each release (in case you need to roll back), and shift all the traffic to the most recent one.
  • To summarize, CodePipeline will glue everything together. It’ll automatically start a build using CodeBuild whenever anything is pushed to your CodeCommit/GitHub repository. Then when your build is completed and the artifacts are uploaded, it will kick off CloudFormation to update and swap out the application in the Lambda.

Stage 1: Creating the git repositories using CodeCommit

Since we’re using AWS services, we might as well keep everything centralized here. The first step is to start our git repository which can be done using Amazon CodeCommit, Amazon S3 or an external source, such as Github. For the current purpose, we will start one using CodeCommit. If you have more than one source, both repositories should be created in this stage.

Make sure to install aws-serverless-express to be able to server your NodeJS API as a serverless function.
git initgit remote add origin <clone url>
git add -Agit commit -m “Initial commit”git push origin master

Stage 2: Build

The next step is build. We will instruct CodeBuild to package our artifacts into a compressed file uploaded on an S3 bucket. The next step will pick it up and deploy. To start, create a new build project.

CodeBuild — Specify the name of your repository. In this case, we’re using the internal CodeCommit.
CodeBuild — Specify the service role name.
CodeBuild — Specify the Artifact output. In this case, we’re using an AWS S3 bucket.
The buildspec file will define the steps required to build the project.
  • Generating and build or packaging everything into production ready
  • An optional extra stage to run your tests
CodeBuild — The buildspec YAML file.

Stage 3: Deploy

Having a build stage ready to output our artifact(s), the next stage is to prepare their deployment. Now here’s the tricky part.

3.1. CloudFormation Template Definition

Back to the stage logic; in sum, our template requirements include the definition of a resource that receives a build bundle as input and creates a new lambda function with it. The ideal use case for this function would be:

  1. Gradually shifts customer traffic to the new version until you’re satisfied that it’s working as expected, or you roll back the update.
  • API Gateway: The API Gateway will be responsible for exposing your lambda function as RESTful API to the Internet (a bridge).
  • Invoke permission: This resource is required to guarantee the gateway can actually invoke and pass the requests to the lambda.
AWS SAM Template for the definition and deployment of the lambda function

3.2. Closing the Pipeline

Finally, after creating each stage individually, we have all the dependencies required to start creating a new pipeline.

Pipeline — Setup the name and respective service role
Pipeline — Define the source repository
Pipeline — Describe the build step using CodeBuild from previous stage 2.
Deployment — Create a new action with type “Create or replace a change set”. Make sure that the File name under the template tab matches the SAM template you’ve generated in stage 3.1.
Deployment — Since the template created in stage 3.1. will contain an action to generate a new role, we must explicitly acknowledge the IAM capability.
  • PermissionsAWSLambdaExecute
  • Role namecfn-lambda-pipeline
Deployment — This last action is literally just to execute the change-set.
Are we there yet?? YES!

Closing Remarks

CodePipeline is complex to setup. It took us a few weeks and AWS support calls to actually reach a stage where we weren’t updating it anymore. Before using such service, we were confident that keeping our development operations in an AWS managed fashion would ease our life.

References

Software Engineer