Using AWS CodeCommit and Lambda for automatic code deployment to S3-bucket

Nov 13, 2018 · 6 min read

AWS S3 buckets can be used for hosting static websites. Setting up such a bucket via the AWS web console takes just a few minutes. However, uploading and maintaining the code can be little tedious. Especially, if you want to use a version control like Git.

This process can be streamlined by using AWS CodeCommit and an AWS Lambda function. CodeCommit provides source/version control servers which allows you to create Git repositories similar to GitHub. But in contrast to GitHub, the repos can be set to private even in the free version. The Lambda function can be build in such a way that it is triggered when a new commit is pushed. In this case, it should take the code from the repo and upload it to the S3-bucket. This means if a new feature is ready for production we just need to commit and push the code. The rest is handled by the Lambda function.

Let’s have a look how we can accomplish that. It is actual not too complicated. (I assume that you already have an AWS account and are familiar with the basic AWS features.)

S3 Bucket for Website Hosting

First, we open the AWS S3 console and create a new S3 Bucket that is used to host our website. We have to provide a unique name for the bucket but otherwise use the default settings. Afterwards, we open the bucket, go to the Properties-tab and activate Static website hosting. We select ‘Use this bucket to host a website’ and define the index document of our website (e.g. index.html).

Activate Static Website Hosting

The ‘Static Website Hosting’-card also provides the endpoint URL of our bucket. If we open this URL in the browser, our index document should be opened. However, this will only work if we grant public access to our bucket. To accomplish that we open the Permissions-tab and define the following Bucket Policy (don’t forget to change the bucket-name).

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucket-name/*"
}
]
}

By the way, you can also link the S3-bucket to a custom domain using AWS Route53. You can find a detailed description here.

CodeCommit Repository

Next, we create a new Git repository with AWS CodeCommit. Again, we just have to provide a name for the repo. The rest is handled by AWS.

We can configure our local Git client to connect to this repo. If you use phpStrom for web development as I do, you can also use its internal version control service. To establish a connection the HTTPS-URL of the repository is required. This can be found when the repo is opened in the AWS console (see the image below).

Clone the HTTPS-URL of the new repo

Furthermore, we have to create a new IAM user which we will use for signing in from our Git client. We go to the AWS IAM console, select Users from the left menu and click on ‘Add user’. Let’s name this user codeCommit and select ‘Programmatic access’. On the next page, we select ‘Attach existing policies’ and search for the policy: AWSCodeCommitFullAccess. This policy allows our new user to read and write the CodeCommit repositories. We attach it to the user and finish the creation. Next, we open the new user in the AWS console and go the ‘Security credentials’-tab. We scroll down to the ‘HTTPS Git credentials for AWS CodeCommit’-section and generate a Git username and password. This are the credentials we will actually use to connect the Git client to our new repo. Afterwards, we can already push the first commit to the repository.

Lambda Function

The last step is setting up the Lambda function. The Lambda function which will be triggered when a commit is pushed to our repository uploads the new code to the S3-bucket.

Before we build the Lambda function, we have to define a new IAM role that allows the Lambda function to access the S3-bucket as well as the CodeCommit repositories. We go to the AWS IAM console, select Roles from the left menu and click on ‘Create role’. We select Lambda from the section ‘Choose the service that will use this role’ and click Next. We attach two policies to the role: AWSLambdaExecute and AWSCodeCommitReadOnly. Then we define a name for the role and finish the creation.

Afterwards, we go to the AWS Lambda console and create a new function. The wizard asks for a name for our new function, a runtime (we use Python 3.6) and a role (we use the role we just created). On the next page, we setup the actual Lambda function.

First, we define a trigger that starts the function. We choose CodeCommit form the ‘Add triggers’-list in the Designer-section. A new trigger is added. If we select it, we can fine-tune its properties in the ‘Configure triggers’-section below. We have to select the name of our repository and define a trigger name. We also have to choose an event the trigger is listen for. We use ‘Push to existing branch’ and select the master branch form the next dropdown. (In order to see any branches in the dropdown you first have to push at least one commit to the repository.) Then we click Add.

Create a CodeCommit trigger to start the Lambda function
Select the Lambda function and add the function code

Next, we select the actual Lambda function in the Designer-section. This enables us to define source code in the ‘Function code’-section below. You can find the code which we use below and on GitHub. It is written in Python. (The code was actually inspired by this stackoverflow post by SamvL.) What the code basically does is creating a list of all files in the repo, reading their contents as binary large objects (BLOBs) and uploading them to the S3-bucket (the folder structure is preserved).

import boto3
import os
import mimetypes
# basically returns a list of a all files in the branch
def get_blob_list(codecommit, repository, branch):
response = codecommit.get_differences(
repositoryName=repository,
afterCommitSpecifier=branch,
)
blob_list = [difference['afterBlob'] for difference in response['differences']]
while 'nextToken' in response:
response = codecommit.get_differences(
repositoryName=repository,
afterCommitSpecifier=branch,
nextToken=response['nextToken']
)
blob_list += [difference['afterBlob'] for difference in response['differences']]
return blob_list# lambda-function
# triggered by changes in a codecommit repository
# reads files in the repository and uploads them to s3-bucket
#
# ENVIRONMENT VARIABLES:
# s3BucketName
# codecommitRegion
# repository
# branch
#
# TIME OUT: 1 min
#
# EXECUTION ROLE
# lambda-codecommit-s3-execution-role (permissions: AWSCodeCommitReadOnly, AWSLambdaExecute)
#
def lambda_handler(event, context):
# target bucket
bucket = boto3.resource('s3').Bucket(os.environ['s3BucketName'])
# source codecommit
codecommit = boto3.client('codecommit', region_name=os.environ['codecommitRegion'])
repository_name = os.environ['repository']
# reads each file in the branch and uploads it to the s3 bucket
for blob in get_blob_list(codecommit, repository_name, os.environ['branch']):
path = blob['path']
content = (codecommit.get_blob(repositoryName=repository_name, blobId=blob['blobId']))['content']
# we have to guess the mime content-type of the files and provide it to S3 since S3 cannot do this on its own.
content_type = mimetypes.guess_type(path)[0]
if content_type is not None:
bucket.put_object(Body=(content), Key=path, ContentType=content_type)
else:
bucket.put_object(Body=(content), Key=path)

The names of the S3-bucket, the CodeCommit repository, the branch and the CodeCommit region are not hardcoded in the function code but have to be defined via Environment variables. This way, the parameters can be easily adapted without touching the code. The image below shows how the variables have to be configured.

Setting up the environment variables

Finally, we go to the ‘Basic Settings’-section and increase the Timeout to 1 min since the Lambda function might run for some time if the repo contains more files.

That’s it. The Lambda function is ready for testing. (Don’t forget to save it). To test it, you can either create a new test in the Lambda-console or just push a commit to the repo.

I hope this tutorial helps you. Let me know what you think in the comments below.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade