Simple CICD Pipeline for Cloudformation

I worked with a client who had decided to move to using infrastructure as code, and implement Cloudformation. Unforunately, the deployment method involved emailing the templates between each other when a deployment was neccesary — you can imagine why this is a bad idea — engineers ended up with the wrong version, or applied development enviroments to production.

In order to resolve this, I setup a simple CICD pipeline to enforce commits to a repo, validate the templates and produce a Cloudformation changeset.

I will use three AWS technologies to achieve this

  • Codecommit — code repo service hosted by AWS in order to store the Cloudformation templates
  • Codebuild — code build service hosted by AWS to build and validate commited code to the Codecommit repo
  • Codepipeline – the “glue” in between the two former platforms

The pipeline is fairly basic. It does not enforce the intital commit to Cloudformation, but will enforce further changes with a Changeset. Hopefully it will get you started and you can expand on it.

Cloudformation Initial Commit

We will deploy a Cloudformation template which is modified from Amazon’s examples

The original is here

https://s3-us-west-2.amazonaws.com/cloudformation-templates-us-west-2/EC2InstanceWithSecurityGroupSample.template

And my modified version is here

Above, I’ve modified the template to add an egress rule, to pass our CICD tests — more on this later.

I will assume you have AWS credentials setup on your system.

Make the inital deployment of the Cloudformation template, replacing KeyName with an existing key pair.

You may want to change SSHLocation from 0.0.0.0/0 to your public IP, before doing this.

aws cloudformation create-stack --template-body file://cf.json --stack-name cf-test --parameters ParameterKey=KeyName,ParameterValue=mykeypair ParameterKey=InstanceType,ParameterValue=t2.small

Codecommit

Firstly, we must create a repo within Codecommit to host our code.

Navigate to Codecommit on the AWS console and select “Create repository”

Skip notifications for now

AWS will provide instructions to clone the repo to your local system — I will assume here that you have already setup access keys

Take note if you are using OSX, Keychain will need the access key entry deleted every time you make changes, or disabled completely.

Once you have cloned the repo, we must add

Copy the Cloudformation template that I modified above, save as cf.json within the repo directory.

Add a new file called buildspec.yml, as per below

This file instructs Codebuild to run the content contained within it — specifically we are

  • Installing cfn-nag https://github.com/stelligent/cfn_nag
  • Using AWS CLI to validate the JSON content of the Cloudformation template
  • Running cfn-nag against the template
  • If this passes, we create a changeset to be reviewed

Commit the changes to the repo

git add .
git commit -m "First commit"
git push

Codecommit

Navigate to Codebuild on the AWS console and click “Create Project”

There are too many options to screenshot here so I will list them

  • Project Name : Friendly name, e.g cf-test
  • Source Provider: AWS CodeCommit
  • Repository : Choose the repo you created earlier
  • Enviroment Image : Ubuntu
  • Runtime : Ruby (required for cfn-nag dependancies)
  • Runtime Version: ruby:2.5.1
  • Build Spec : Use the buildspec.yml in the source code root directory

Anything else is default, save the template

CodePipeline

Navigate to CodePipeline in the AWS console and create a new pipeline

  • Name : cf-test
  • Source Provider : AWS Codecommit
  • Repository Name: Codecommit repo created earlier
  • Branch Name : Master
  • Build Provider : AWS CodeBuild
  • Project Name: AWS CodeBuild project created earlier
  • Deployment Stage: No Deployment
  • Role Name : Create a role

IAM

We need to update the role previously created, to add permissions to edit Cloudformation

Navigate to AWS Console -> IAM -> Roles and search for the created role “codebuild-cf-test-service-role

Open the role and select “Add inline policy”

Select the JSON editor tab and paste in the following

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"cloudformation:CreateUploadBucket",
"cloudformation:EstimateTemplateCost",
"cloudformation:ListExports",
"cloudformation:PreviewStackUpdate",
"cloudformation:ListStacks",
"cloudformation:ListImports",
"cloudformation:DescribeAccountLimits",
"cloudformation:DescribeChangeSet",
"cloudformation:ValidateTemplate"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "cloudformation:*",
"Resource": "arn:aws:cloudformation:*:*:stack/*/*"
}
]
}

Review, name the policy “CodebuildCloudformation” and save

Making some changes

In the Cloudformation working directory, create a new branch

git checkout -b cf-test

Open cf.json with a text editor of your choice and change the line highlighted below from t2.small to t2.nano

"InstanceType" : {
"Description" : "WebServer EC2 instance type",
"Type" : "String",
"Default" : "t2.nano",
"ConstraintDescription" : "must be a valid EC2 instance type."
},

Save, create a branch and commit changes

git add cf.json
git commit -m "First commit"
git push --set-upstream origin master

Navigate to Codecommit on the AWS console and create a pull request

Set the Destination as Master and Source as cf-test

Set a title for the merge request and review the changes, select “Save” and then “Merge”

Navigate to CodeBuild, select your project and under “Build history” you will see the build “in progress”

If we click on the Build Run, we can monitor the build live

Here is a sample build run — note that this build failed, as initially I had left the egress rule out, meaning cfn-nag failed and exited code 1, failing the build.

The error from cfn-nag is marked in bold.

Installing cfg-nag
[Container] 2018/10/02 22:02:13 Running command gem install cfn-nag
Successfully installed kwalify-0.7.2
Successfully installed cfn-model-0.1.26
Successfully installed jmespath-1.3.1
Successfully installed little-plugger-1.1.4
Successfully installed multi_json-1.13.1
Successfully installed logging-2.2.2
Successfully installed netaddr-1.5.1
Successfully installed trollop-2.1.3
Successfully installed cfn-nag-0.3.54
9 gems installed
[Container] 2018/10/02 22:02:14 Phase complete: PRE_BUILD Success: true
[Container] 2018/10/02 22:02:14 Phase context status code:  Message:
[Container] 2018/10/02 22:02:14 Entering phase BUILD
[Container] 2018/10/02 22:02:14 Running command echo Testing CF Template
Testing CF Template
[Container] 2018/10/02 22:02:14 Running command aws cloudformation validate-template --template-body file://cf.json
{
"Description": "AWS CloudFormation Sample Template EC2InstanceWithSecurityGroupSample: Create an Amazon EC2 instance running the Amazon Linux AMI. The AMI is chosen based on the region in which the stack is run. This example creates an EC2 security group for the instance to give you SSH access. **WARNING** This template creates an Amazon EC2 instance. You will be billed for the AWS resources used if you create a stack from this template.",
"Parameters": [
{
"NoEcho": false,
"Description": "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"ParameterKey": "KeyName"
},
{
"DefaultValue": "0.0.0.0/0",
"NoEcho": false,
"Description": "The IP address range that can be used to SSH to the EC2 instances",
"ParameterKey": "SSHLocation"
},
{
"DefaultValue": "t2.nano",
"NoEcho": false,
"Description": "WebServer EC2 instance type",
"ParameterKey": "InstanceType"
}
]
}
[Container] 2018/10/02 22:02:15 Running command /usr/local/bundle/bin/cfn_nag_scan --input-path cf.json
------------------------------------------------------------
cf.json
------------------------------------------------------------------------------------------------------------------------
| FAIL F1000
|
| Resources: ["InstanceSecurityGroup"]
|
| Missing egress rule means all traffic is allowed outbound.  Make this explicit if it is desired configuration
Failures count: 1
Warnings count: 0
[Container] 2018/10/02 22:02:16 Command did not exit successfully /usr/local/bundle/bin/cfn_nag_scan --input-path cf.json exit status 1
[Container] 2018/10/02 22:02:16 Phase complete: BUILD Success: false
[Container] 2018/10/02 22:02:16 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: /usr/local/bundle/bin/cfn_nag_scan --input-path cf.json. Reason: exit status 1
[Container] 2018/10/02 22:02:16 Entering phase POST_BUILD
[Container] 2018/10/02 22:02:16 Phase complete: POST_BUILD Success: true
[Container] 2018/10/02 22:02:16 Phase context status code:  Message:
[Container] 2018/10/02 22:02:16 Expanding base directory path: .
[Container] 2018/10/02 22:02:16 Assembling file list
[Container] 2018/10/02 22:02:16 Expanding .
[Container] 2018/10/02 22:02:16 Expanding artifact file paths for base directory .
[Container] 2018/10/02 22:02:16 Assembling file list
[Container] 2018/10/02 22:02:16 Expanding cf.json
[Container] 2018/10/02 22:02:16 Found 1 file(s)
[Container] 2018/10/02 22:02:16 Phase complete: UPLOAD_ARTIFACTS Success: true
[Container] 2018/10/02 22:02:16 Phase context status code:  Message:

So if your build completes, it’s time to check for a change set in the Cloudformation console.

Hopefully if the build completed successfully, you will see a change set under the deployed cf-test template we deployed earlier.

If we open the change, we will see that under “Changes”, the EC2 instance is being replaced as we are resizing to t2.nano.

This really is why change sets are very important – before we deploy the infrastructure, we can understand the impact.

To deploy the changes, hit “Execute”, and the instance will be resized.

Summary

So what does this achieve? Firstly it promotes committing infrastructure changes to a version control, and secondly creates a pull request and Cloudformation change set for review. A colleague could peer review the pull request, before a senior engineer executes the changeset.

Any changes to the infrastructure can be tracked via the git repo, and reverted via Cloudformation.

Additionally we have two simple tests within the pipeline – one to validate the syntax and another to check for compliance failures such as opening ports to the internet, or unencrypted EBS volumes.

Whilst this is a simple pipeline, I hope this article will help you implement some key controls for your infrastructure – much of this can be achieved within AWS Free Forever tier, without the need of a EC2 Instance running, say, Jenkins.