How to apply CI/CD by using GitHub, CodeBuild, CodePipeline and ECS

Khoa Nguyen
Nov 22, 2018 · 4 min read

This is a demo which I want to show you a basic usage of CodeCommit, CodeBuild, CodePipeline and ECS.

All source code is uploaded on Github

Before you go though the detail, I will explain a basic concept of these services. However, I think you can access the AWS Docs directly to understand deeply.

CodeBuild is a build service. We can define the environment such as Python, Ruby , Docker and Go. For instance, our application is written by Python. And we need to an environment to run unit test when a developer commit code. If the unit test fails , there is an alarm to notify other developers. We can configure some tasks such as building artifacts, building docker images and deploy processes.

CodePipeline is a service which combine CodeBuild, CodeDeploy, and Source by order. For example, in step one, we can configure where contains source code such as Github, CodeCommit , S3 or Bitbucket. In step 2, we can configure CodeBuild to run Unit Test. In step 3, we can configure CodeBuild to build artifacts . In step 4, we can configure build docker images. In step 5, we can configure CodeDeploy to deploy the application.

ECR is like a docker hub where we can save Docker images. We can pull and push docker images from ECR.

ECS is a docker orchestration tools. It has a range of servers which is called Workers. We can scale up and scale down containers, and also workers ( servers ). It has a user interface where we can manage our infrastructure resources such as Ram, CPU and Container merits easily and effectively. Moreover, it can integrate with other services like CloudWatch Logs for logging.

In this tutorial, I assume that you have basic knowledge of AWS services like ECR, ECS, and Docker. I will focus on CodePipeline, CodeBuild to demonstrate how to apply CI/CD with AWS services.

So this is a basic CI/CD flow to demonstrate how to deploy a Flask application when developers commit the source code.

  1. The developer will commit code on Github.
  2. The CodePipeline will poll source code when there is any changes.
  3. The CodeBuild will run unit test, build a docker image, and push the image to ECR.
  4. The CodeBuild will execute a deployment script to deploy application on ECS.

Flask Application

Create a “Hello World” Flask application, all source code is on GitHub

➜  web tree .
.
├── Dockerfile
├── app.py
├── buildspec.yml
├── deploy.sh
├── docker-compose.yml
├── requirements.txt
├── service.json
├── task.json
└── test_basic.py

buildspect.yml: This file is used for building docker image by CodeBuild.

deploy.sh: This file is deployment script which is used by CodeBuild

task.json: This is the ECS task template definition.

service.json: This is the ECS service.

Build the infrastructure by Terraform

In Terraform, It’s contain some modules such as nat gateway, network, vpc and subnet to create a new VPC.

In ECS module, It will create ECS cluster, ECR, CodeBuild, CodePipeline and other IAM roles.

The main point of this tutorial is focusing on applying CI/CD with CodePipeline.

Take a look at codepipeline resource by Terraform. It contains 3 stage.

  • The stage one, we configure CodePipeline to poll code from Github.
  • In stage two, we use CodeBuild to run unit test and build docker image.
  • In stage three, we use CodeBuild to execute the scripts to deploy application.

Stage 1: Poll source code from Github.

We should define Github Token, Repo and Branch. CodePipeline will base on these information to get the source code.

Step2 : Build Docker Image.

We will dive into CodeBuild to run Unit Test and build Docker Image now.

In this CodeBuild, we need to indicate some parameters such as computer_type, image and type and you need to give the path of buildspec.yml

The CodeBuild will run step by step. Take a look at buildspec.yml in web folder.

buildspec.yml contains a set of commands. For example, this file has commands to build docker image and push it to ECR.

CODEBUILD_RESOLVED_SOURCE_VERSION : This variable is a git commit id.

Everytime, a developer commit code on Github, the CodeBuild will build an image and tag the image with git commit hash, then push the image to ECR.

Step3: Deploy application on ECR.

We pass some environment variables into deploy.sh script. The script will create ECS task definition and ECS service.

Conclusion

To understand how modules work together, you can check out my Github repo.

I welcome any feedbacks from you. You can leave the comment below. I will answer when i get a chance.

Khoa Nguyen

Written by

I’m a DevOps engineer. I like travelling, surfing and meeting new people.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade