Serverless CICD Pipeline to Delivery Apps to AWS Fargate — A Serverless Container Platform

Haimo Zhang
May 5 · 7 min read

AWS Fargate is a “serverless” technology for AWS ECS. It allows users to focus on applications instead of managing underline infrastructures.

Amazon Elastic Container Service(ECS) has two modes: Fargate launch type and EC2 launch type. There are several benefits when using Fargate launch type.

(1) No EC2 or Cluster to manage. Using AWS Fargate, no need to worry about EC2 instance type, EC2 auto scaling, etc., all these are going away from our concern.

(2) Seamless scaling, using Fargate, we only need to define application requirements ( CPU, memory, etc.), AWS Fargate manages all the scaling and infrastructure needed to run containers in a highly-available manner.

(3) Seamless integrate with ECS, Just need to define task definition and service the same way as for ECS, AWS Fargate launches and manages containers for us.

Below is the architecture diagram for the CICD pipeline to be built in this blog.

Here is high level overview of the procedures involved.

First, creating two Fargate clusters for two environments, one is for UAT, one is for Prod.

Also need to create the following AWS resources.

AWS ECR Repo

CodeCommit Repo

Application Load Balancer

Second, uploading application files together with the following help files to CodeCommit repo.

buildspec.yml

Dockerfile

ecs-taskdefinition-service-uat.json

ecs-taskdefinition-service-prod.json

Third, creating a build project using CodeBuild, this will run “docker build” to build container image, and send the image to ECR Repo.

Lastly, creating a pipeline using CodePipeline console to automate the source, build and deploy stages, and deploy cloudformation template to provision container and service on AWS Fargate.

Let’s start an sample setup.

(1) Setup AWS Fargate

Going to ECS console, and choose “create cluster”.

Choose “Networking only, powered by AWS Fargate”

Select “Create VPC” and fill in CIDR block, subnet1 and subnet2, click “create”, we can see the ECS cluster created.

(2) Create ECR Repository

In ECS console, choose “Repositories” on left panel, and click “Create Repository”

click “Next Step”

(3) Create Application Load Balancer.

Login AWS EC2 console, choose “load balancers” from left panel, click “Create Load Balancer”. select “Create” from Application Load Balancer. put Name, and choose the VPC just created from step (1).

Create a new security group, for this demo, we have port 8080 and 80 open.

Create a new target group, for fargate, the target type is ip

Then choose “Next: Register Targets”

(5) Editing helper files.

buildspec.yml

version: 0.2env:
variables:
AWS_DEFAULT_REGION: "us-east-1"
AWS_ACCOUNT_ID: "199xxxxxxxxx"
IMAGE_REPO_NAME: "fargate-ecr"
IMAGE_TAG: "latest"
phases:
pre_build:
commands:
- sudo apt-get update -y && apt-get install jq -y
- echo Logging in to Amazon ECR...
- $(aws ecr get-login --region $AWS_DEFAULT_REGION --no-include-email)
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t $IMAGE_REPO_NAME .
- docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker image...
- docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
- tmp=$(mktemp) && SHA=$(docker inspect $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:latest | jq -c '.[0].RepoDigests' | sed 's/[][]//g' | sed 's/"//g') && jq -r --arg SHA "$SHA" '.Resources.TaskDefinition1.Properties.ContainerDefinitions[0].Image=$SHA' ecs-taskdefinition-service-uat.json > "$tmp" && mv -f "$tmp" ecs-taskdefinition-service-uat.json
- tmp=$(mktemp) && SHA=$(docker inspect $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:latest | jq -c '.[0].RepoDigests' | sed 's/[][]//g' | sed 's/"//g') && jq -r --arg SHA "$SHA" '.Resources.TaskDefinition1.Properties.ContainerDefinitions[0].Image=$SHA' ecs-taskdefinition-service-prod.json > "$tmp" && mv -f "$tmp" ecs-taskdefinition-service-prod.json
artifacts:
files:
- ecs-taskdefinition-service-uat.json
- ecs-taskdefinition-service-prod.json
discard-paths: yes

ecs-taskdefinition-service-uat.json (similar settings for ecs-taskdefinition-service-prod.json)

{
"AWSTemplateFormatVersion":"2010-09-09",
"Parameters":{
"Tag":{
"Type":"String",
"Default":"latest"
},
"DesiredCount":{
"Type":"Number",
"Default":1
},
"Cluster":{
"Type":"String",
"Default":"fargate-uat"
},
"Repository":{
"Type":"String",
"Default":"fargate-ecr"
}
},
"Resources":{
"Service":{
"Type":"AWS::ECS::Service",
"Properties":{
"Cluster":{
"Ref":"Cluster"
},
"DesiredCount":{
"Ref":"DesiredCount"
},
"LoadBalancers":[
{
"ContainerName":"fargate-container",
"ContainerPort":"8080",
"TargetGroupArn":"arn:aws:elasticloadbalancing:us-east-1:199xxxxxxxxx:targetgroup/tg-fargate-uat/b1f837d57f2c8c90"
}
],
"TaskDefinition":{
"Ref":"TaskDefinition1"
},
"LaunchType":"FARGATE",
"NetworkConfiguration":{
"AwsvpcConfiguration":{
"AssignPublicIp":"ENABLED",
"SecurityGroups":[
"sg-05073135deb4acacb"
],
"Subnets":[
"subnet-0037ad17a10499a17",
"subnet-057694008680b185d"
]
}
}
}
},
"TaskDefinition1":{
"Type":"AWS::ECS::TaskDefinition",
"Properties":{
"RequiresCompatibilities":[
"EC2",
"FARGATE"
],
"NetworkMode":"awsvpc",
"Cpu":"256",
"ExecutionRoleArn":"arn:aws:iam::199xxxxxxxxx:role/ecsTaskExecutionRole",
"TaskRoleArn":"arn:aws:iam::199xxxxxxxxx:role/ecsTaskExecutionRole",
"Memory":"512",
"ContainerDefinitions":[
{
"Name":"fargate-container",
"Image":"199xxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/fargate-ecr:latest",
"PortMappings":[
{
"ContainerPort":"8080",
"HostPort":"8080"
}
],
"Essential":"true"
}
]
}
}
}
}

(6) Setting up CodeCommit and push application code and helper files to CodeCommit

Go to CodeCommit console, and create a repository

After click “Create repository” then clone the repo to laptop local git environment.

Get application code into this folder and push to CodeCommit Repo.

And the repo is showing the uploaded files.

(7) Creating a CodeBuild project for the build stage

Go to the AWS console, and click on CodeBuild.

click create project and put info as below and click save.

  • Project name: fargate-build
  • Source provider: AWS CodeCommit
  • Repository: fargate-repo-simple
  • Environment image: Use an image managed by AWS CodeBuild
  • OS: Ubuntu
  • Runtime: Docker
  • Version: aws/codebuild/docker:1.12.1
  • Build specification: Use the buildspec.yml in the source code root directory
  • Artifact Type as below
  • Service Role: Create a service role in your account (later, we will attach the EC2 Container Registry access Policy with this role)

Also select VPC as the one created in fargate creation stage (change to “NO VPC” otherwise the build failed.)

click continue and then click save.

the build job starts.

(3) Granting ECR access permission to CodeBuild service.

Search codebuild in role section of IAM console, then click the one just created for this build and attach the policy “AmazonEC2ContainerRegistryFullAccess”

(4) Editing buildspec.yml, Dockerfile, and cloudformation template

Update the following parameters in buildspec.yml

AWS_DEFAULT_REGION: “us-east-1” (You can choose your own region)AWS_ACCOUNT_ID: “YOUR AWS ACCOUNTID”

IMAGE_REPO_NAME: “fargate-ecr”

ECR REPO URI: 199476098484.dkr.ecr.us-east-1.amazonaws.com/fargate-ecr

update CF template file ecs-taskdefinition-service.json

ClusterName: fargate-cluster
Repository: fargate-ecr
Image: xxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/fargate-ecr:latest

Then push the change to CodeCommit repo.

(5) Creating a CodePipeline using CodeCommit, CodeBuild, and CloudFormation

Go to AWS console, and click CodePipeline

Then click on Next step, choose as below.

After review, click create pipeline, and this will be illustrated in the pipeline stages.

Summary

The setup demonstrated a powerful CICD pipeline uses all AWS services, no EC2 or cluster to manage, no third party tools to install.

This serverless CICD Pipeline is to continuously build and delivery. It also can goes cross different environments after automating testing and approval procedures.

The target AWS Fargate is serverless container platform. It allows us to focus on application and no worry about underline infrastructure. It can seamlessly scale out performance and seamlessly integrate with other AWS services.

This is a simplified CICD pipeline, in real situation, we can plugin multiple testing stages in the AWS CodePipeline which is already integrated with multiple third party testing services.

Faun

The Must-Read Publication for Aspiring Developers & DevOps Enthusiasts

Haimo Zhang

Written by

AWS, GCP, Microservices, Kubernetes, Istio, Kafka

Faun

Faun

The Must-Read Publication for Aspiring Developers & DevOps Enthusiasts