Auto Deploy to AWS Fargate with Docker-Compose and ECS Params.

Let us deploy.

In this article, I will attempt to explain how you can take your deployment strategy from manual to auto, especially when dealing with AWS-ECS Fargate. The cool part is we are making use of the docker-compose utility, so let’s see how we can achieve this. Our focus here will be on auto-deployment and not what AWS Fargate is. To know about AWS Fargate, please reference the docs here and the resources section at the end of this article.

My guess here is there are a couple of ways to accomplish this, but hey let’s consider one approach amongst others.

GitHub Repo: https://github.com/andela-sjames/rainbowtext

Updated: 11 February, 2020.

NB: Prerequisite.

1. You should create an IAM user from your AWS dashboard to programmatically have access to ECS, ECR, Cloudwatch and IAM. This will be your policy, I created one for myself and called it AmazonECSTaskExecutionRolePolicy , you should use this name for consistency with this tutorial.
2. You should have the AWS CLI and ECS CLI tool installed on your machine
3. You should have a Python virtual environment activated with the requirements installed for the project.

# Create an AWS IAM user
https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-started_create-admin-group.html
# set up ecs-cli quick guide by aws
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI_installation.html
# set up AWS CLI
https://docs.aws.amazon.com/comprehend/latest/dg/setup-awscli.html
# Youtube Video on AWS IAM if you prefer
https://www.youtube.com/watch?v=DXNS-EP9sXM Part 1
https://www.youtube.com/watch?v=z9MOPMxnCjY Part 2
# Pyenv virtualenv
https://github.com/pyenv/pyenv-virtualenv

The Sample App

From the Github link referenced above, you can run the sample app in development mode to see what you are about to push to AWS. Clone the repository using the command git clone git@github.com:andela-sjames/rainbowtext.git

Once cloned, you need to follow the instructions from the readme to set up and configure AWS locally.

To be clear and explicit about the environment variables you will need. Create a .env file in the scripts folder of the project and add the following lines to the file.

# Project name needs to match the docker image prefix, which is by default the same as the project root dir name
export AWS_PROJECT=rainbowtext
export AWS_ROLE=<YOUR_AWS_ROLE_NAME>

export AWS_ACCOUNT_ID=<YOUR_AWS_IAM_ACCOUNT_ID>
export AWS_REGION=<YOUR_AWS_REGION>

export AWS_ACCESS_KEY_ID=<YOUR_AWS_IAM_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<YOUR_AWS_IAM_SECRET_ACCESS_KEY>

export DOCKER_COMPOSE_YML_INPUT=docker-compose.yml
export DOCKER_COMPOSE_YML_OUTPUT=docker-compose.ecs.yml

export ECS_PARAMS_INPUT=ecs-params.yml
export ECS_PARAMS_OUTPUT=ecs-params.ecs.yml

Next, run source scripts/.env to load your environment variables. The Readme says this and more. With that done we are ready to run this app locally using the command docker-compose build && docker-compose up

You should see something similar to the image below

when you navigate to http://localhost.

Pre-Deploy

A couple of things we should know before we run the deploy script. The project makes use of Nginx as a reverse proxy for production; you can check out this blog post I wrote on using Nginx as a proxy server to clarify the use case. The Deploy script does three basic things using three files

• scripts/login_ecr.sh: It configures AWS on your machine with a custom profile and logs into ECR.
• scripts/config_ecr.py: It creates a repo on ECR and uploads created and tagged images to ECR.
• scripts/setup_ecs.sh rainbowtext: Setup ECS and deploy to ECS Fargate.

Login to ECR:

Amazon Elastic Container Registry (ECR) is a fully-managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images. Our script login_ecr.sh

• Sets up AWS locally on your shell using the rainbowtextprofile
• Gets ECR logging details
• Authenticate via docker

# login to ECR
echo 'logging into ECR'
aws configure set aws_access_key_id "$AWS_ACCESS_KEY_ID" --profile ${AWS_PROJECT}
aws configure set aws_secret_access_key "$AWS_SECRET_ACCESS_KEY" --profile ${AWS_PROJECT}
aws configure set default.region ${AWS_REGION} --profile ${AWS_PROJECT}
aws configure set default.output json --profile ${AWS_PROJECT}

# get ecr logging details.
token=$(aws ecr get-authorization-token)
registry=$(echo "$token" | jq -r .authorizationData[].proxyEndpoint | sed -e 's|https://||g')
pass=$(echo "$token" | jq -r .authorizationData[].authorizationToken | base64 -d | cut -d: -f2)

# login via docker
echo "$pass" | docker login -u AWS --password-stdin "$registry"
echo 'aws auth successful'

Configure ECR

The scripts/configure_ecr.py file does a couple of exciting things courtesy of this snippet I found on GitHub, I modified the script to do my bidding.

It does five essential things for us( Please reference the GitHub repo)

1. It creates the ECR repo on AWS
2. It re-tags the built production images for ECR
3. It pushes the newly tagged images to ECR
4. It updates our docker-compose service by adding AWS ECS specific parameters to the file for logging.

#... snippet from confir_ecr.py
if service_name == "server":
    service["logging"] = {
        'driver': 'awslogs',
        'options': {
            'awslogs-group': 'rainbowtext',
            'awslogs-region': 'us-east-1',
            'awslogs-stream-prefix': 'rainbowtext-server'
        }
    }
if service_name == "nginx":
    service["logging"] = {
        'driver': 'awslogs',
        'options': {
            'awslogs-group': 'rainbowtext',
            'awslogs-region': 'us-east-1',
            'awslogs-stream-prefix': 'rainbowtext-nginx'
        }
    }

5. It creates a new docker-compose file called docker-compose.ecs.yml that will be used later on to deploy our services to ECS.

The functions in question here are:

create_ecr_repo(services)
re_tag_images(res)
push_to_ecr(res)
update_services(res)
create_deploy_docker_compose_file(output_file)

Setup ECS

The scripts/setup_ecs.sh rainbowtext does the following things

1. It creates the task execution role for ECS using the file task-execution-assume-role.json
2. It creates the ECS cluster and the ECS profile
3. It starts the cluster
4. It Configures the ECS parameters for the ECS cluster using the set_ecs_params.py Python script. This script uses the ecs-params.yml file to configure the security group and subnet on a new file called ecs-params.ecs.ymlfor our AWS ECS newly created cluster programmatically.
5. It deploys to the created ECS cluster.

# deploy to the ecs cluster
ecs-cli compose --file ${DOCKER_COMPOSE_YML_OUTPUT} --ecs-params ${ECS_PARAMS_OUTPUT} --project-name ${project_name} service up --create-log-groups --cluster-config ${project_name}

Deploy?

Not yet. Before we run the deploy script let’s get some things out of the way, the production app runs using the uWSGI webserver. The Dockerfile says a little about it with the command

...dockerfile snippet
ENTRYPOINT [ "uwsgi", "--ini", "uwsgi.ini" ]

You can find the `uwsgi` config for the app in the uwsgi.ini file

[uwsgi]
module = app
callable = app
uid = www-data
gid = www-data
master = true
processes = 5
http = 0.0.0.0:8080
buffer-size = 65535
chmod-sock = 664
vacuum = true
die-on-term = true

To know more about WSGI, please reference this article by Full Stack python and the resources section at the end of this article.

Let Us Deploy

Run the command docker-compose build --build-arg build_env="production" to build the production image first. The build step is vital as the development image for Nginx uses the config below

# Nginx config snippet development
upstream app {
    server server:8080;
}

Which assumes we are running within the context of docker, docker makes use of the Bridge Network Mode for service discovery and auto assigns an IP address to the server:8000 block, e.g. 172.16.20.4:8000.

The Production build assumes we are on ECS Fargate which relies on awsvpc Network Mode, AWS Fargate launched with multiple containers as part of a single task allows each service to communication on a single loopback interface (127.0.0.1) via different ports, as the awsvpc networking mode gives all containers in a task a shared elastic network interface (ENI) to use for communication.

# Nginx config snippet production
upstream app {
    server 127.0.0.1:8080;
}

After building the image for production, we need to activate our Python virtual environment before we deploy using the command ./scripts/deploy.sh

The deployment will take a couple of seconds to finish execution. Perhaps, a minute for a start. Once deployed, we can do a couple of things like view the containers running to scaling the tasks on the cluster via the ECS-CLI command utility.

NB: If anything should go wrong on the first attempt, i.e. if you miss any part of the instructions I have provided at first, No worries all you have to do is to run the command ./scripts/destroy.sh and ignore the error messages, this will remove any stack or ECS-Role that was created at first.

# View the Running Containers on a Cluster
ecs-cli compose --project-name rainbowtext service ps --cluster-config rainbowtext
# Link for Further read up on ecs-cli: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-fargate.html

If everything deploys appropriately, you should see something similar to this logged to your console

(rainbowtext) ➜  rainbowtext git:(master) ./scripts/deploy.sh 
logging into ECR
Login Succeeded
aws auth successful
Waiting for rainbowtext_server push to complete...
The push refers to repository [xxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/rainbowtext_server]
3f9dc670b618: Pushed 
148860cfbda7: Pushed 
24218297f1e8: Pushed 
433d6c1e96ce: Pushed 
fbcd0b99e0d2: Pushed 
a69393ea888c: Pushed 
28cbce3ed862: Pushed 
4906afaa995c: Pushed 
80607578daf8: Pushed 
9797041513df: Pushed 
d9ff549177a9: Pushed 
latest: digest: sha256:883d02d0665cbc8c457cee12d7e0fc7e3d93d5eb9b78c752b709120bf99a13aa size: 2627
Done.
Waiting for rainbowtext_nginx push to complete...
The push refers to repository [xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/rainbowtext_nginx]
19d7b35341c8: Pushed 
ced7d074d469: Pushed 
d2cc9173e763: Pushed 
d7acf794921f: Pushed 
d9569ca04881: Pushed 
cf5b3c6798f7: Pushed 
latest: digest: sha256:429e074043b7c4fa2958060586ac165c533e5aa3228badb6e2f3d78b4535e834 size: 1776
Done.
Wrote new compose file.
COMPOSE_FILE=docker-compose.ecs.yml
{
    "Role": {
        "Path": "/",
        "RoleName": "ecsRainbowtextTaskExecutionRole",
        "RoleId": "AROA3F5H4ERDXOCHCEJHP",
        "Arn": "arn:aws:iam::768614346553:role/ecsRainbowtextTaskExecutionRole",
        "CreateDate": "2019-08-23T13:30:46Z",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Sid": "",
                    "Effect": "Allow",
                    "Principal": {
                        "Service": "ecs-tasks.amazonaws.com"
                    },
                    "Action": "sts:AssumeRole"
                }
            ]
        }
    }
}
INFO[0000] Saved ECS CLI cluster configuration rainbowtext. 
INFO[0000] Saved ECS CLI profile configuration rainbowtext. 
WARN[0000] Skipping unsupported YAML option for service...  option name=container_name service name=nginx
WARN[0000] Skipping unsupported YAML option for service...  option name=depends_on service name=nginx
WARN[0000] Skipping unsupported YAML option for service...  option name=container_name service name=server
WARN[0000] Skipping unsupported YAML option for service...  option name=restart service name=server
INFO[0001] Using ECS task definition                     TaskDefinition="rainbowtext:5"
INFO[0003] Created Log Group rainbowtext in us-east-1   
WARN[0003] Failed to create log group rainbowtext in us-east-1: The specified log group already exists 
INFO[0004] Created an ECS service                        service=rainbowtext taskDefinition="rainbowtext:5"
INFO[0005] Updated ECS service successfully              desiredCount=1 force-deployment=false service=rainbowtext
INFO[0021] (service rainbowtext) has started 1 tasks: (task c5da866c-5952-4b47-90f0-a549b7192220).  timestamp="2019-08-23 13:33:45 +0000 UTC"
INFO[0154] Service status                                desiredCount=1 runningCount=1 serviceName=rainbowtext
INFO[0154] ECS Service has reached a stable state        desiredCount=1 runningCount=1 serviceName=rainbowtext
(rainbowtext) ➜  rainbowtext git:(master)

Note on Accessing the Web.

The deployment scripts create a cluster with an auto-generated default security group. The default rules don’t allow inbound traffic from the Internet and we need to change that after services are up. Run grep security_grp deploy.log to extract the auto-generated security group ID. Go to AWS console AWS -> VPC -> Security Groups and search for that ID. Add an HTTP inbound rule.

View the App

To get the IP address of the app from the AWS console, you should follow the following steps:

• Sign in to your AWS account
• Go to your ECS service tabs

• Locate FARGATE the rainbowtext cluster

• From the cluster page click on the task tab

• On the task tab, click on the task link, this will take you to the task page where you can locate the Network section

• There you will find the Public IP Address of your running task for you to view.

In my case, this is what mine looks like when I navigate to the app running on ECS Fargate.

Explore, see what your task definitions look like and happy deployment in your next project.

Destroy

As we all know a demo app running on AWS will undoubtedly incur some cost, if allowed to stay up for a long time, so when you are done exploring Fargate on ECS it’s advisable to pull down the deployed stack, and that’s where the destroy script comes in handy.

Run ./scripts/destroy.sh and the stack, as well as the logs on cloud watch, will be taken down including the task role defined for the deployment.

This is a summary of what takes place when you run the script.

# cleanup aws log group
aws logs delete-log-group --log-group-name ${AWS_PROJECT}
# clean up ecr
python scripts/del_ecr.py
# clean up ecs cluster
ecs-cli compose --file docker-compose.ecs.yml --ecs-params ecs-params.ecs.yml --project-name ${AWS_PROJECT} service down --cluster-config ${AWS_PROJECT}
ecs-cli down --force --cluster-config ${AWS_PROJECT}
# clean up ${AWS_ROLE} mine is ecsRainbowtextTaskExecutionRole
aws iam detach-role-policy --role-name ${AWS_ROLE} --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy --profile ${AWS_PROJECT}
aws iam delete-role --role-name ${AWS_ROLE} --profile ${AWS_PROJECT}

It’s a wrap

The primary focus of the article was on deploying to Fargate using the docker-compose file right? Well, this is what your generated files should look like when you run the script from your machine

docker-compose.ecs.yml

sample docker-compose.ecs.yml

ecs-params.ecs.yml

sample ecs-params.ecs.yml

So what’s next? Happy deployment folks.

Resources/Further Reading:

Installing the Amazon ECS CLI

Running Flask in production with Docker

Getting Started with Amazon ECS using Fargate

Task Networking in AWS Fargate | Amazon Web Services