Docker-Compose + ECS

Assume that you already know what Docker, and Docker-Compose is; Docker-compose is an awesome tool for local development binding all services together, and the service discovery is extremely easy via the use of link. But when it comes to deployment sometime your docker-compose.yml might not work.

In this post, my goal is to write a note to myself regarding my attempt to deploy my project’s docker-compose.yml to Amazon’s ECS.

Big Picture

So you have docker-compose.yml and you want to put it up on the cloud using Amazon’s ECS. What are the steps that I need to take in order to make that happen?

1. You will need a cluster.
2. You will need an ecs-cli compatibledocker-compose.yml

So these 2 are essentials problem you need to address.

You will need a cluster

Amazon ECS is a free service that govern your AWS resources such as EC2. When you ask ECS to do something for you it will be done in the form of Task and Containers. But these Tasks/Containers need to be executed on some machine right? And that refers to your Cluster.

Luckily AWS provided us a set of command line tools. Such as aws and ecs-cli tools. To install these please refer to this official tutorial.

Once you have configured your profile. It is also important to configure your cluster.

ecs-cli configure --cluster cluster_name --default-launch-type launch_type --region region_name --config-name configuration_name

For ease of understanding you can leave --config-name empty so that it is configured as default cluster.

--default-launch-type if you use EC2 it will create set of EC2 machines for you when your cluster is created. (This step is just configuring there is no cluster created yet).

Now once you have your ecs-cli configured. To start your cluster. Use:

ecs-cli up

This will use your default configuration to create your cluster. Once it is done. You will be able to see your cluster here. Here if you configured your cluster using default-launch-type as EC2 you can see list of EC2 instances created in your AWS console. On the other hand, you can use default-launch-type as FARGATE which is a new AWS service, basically it is EC2 without instances! Find out more here.

Of cause you can shutdown your cluster as well. Simply issue:

ecs-cli down

This will delete your Cluster.

You will need an ecs-cli compatible docker-compose.yml

So if you went through enough links you will realized that docker-compose.yml is not completely supported by ecs service. The most important one is that it doesn’t support docker-compose's build command.

So problem is we will need to build our own docker images first and push it on somewhere.

Luckily I watched this video and he shared his scripts which allow us to automate this process. All you need are a docker’s hub account and some knowledge on python. Here is a the script for copy-and-paste. Feel free to modify it.

https://gist.github.com/peatiscoding/3310d89cb871ac5f414dbccdb1ed8d46

Run this script in your folder that contains docker-compose.yml file this will generate a new file with unique number suffix. Use this file to deploy to ecs instead.

Problem with volume

Before we deploy please note that docker-compose.yml will not work if your docker-compose.yml contains volume that mount to your source directory. (Imagine that someone call your docker-compose up on EC2 machine and there is no source to mount).

To fix this problem you will need to modify your docker file to copy the source code instead of mounting it. (But you can still keep volume node in your main docker-compose.yml as for local development.)

My folder structure is similar to this

docker-compose.yml
app/
 +- Dockerfile
 +- project_code/

Example: app/Dockerfile

FROM php:7.1.0-fpm
RUN apt-get update \
    && apt-get purge zlib1g-dev
... bla bla bla ...
ADD ./project_code /source_codes/

And in my main docker-compose file (Local development file)

version: "2"
services:
  php:
    build:
      context: ./app
    mem_limit: 20000000
    volumes:
      - ./app/project_code:/source_codes/

Once new docker-compose file is created. You should remove the source code volume binding from docker-compose file. (Because the source code is already shipped in the image you have build with the script earlier).

Example of built docker-compose.yml:

services:
  mongo:
  php:
    image: peatisdocking/project_php:1516770224
    mem_limit: 20000000

Notice that I’ve removed volumes node.

That’s it. Your docker file should be ready for deployment. To deploy simply use ecs-cli command as follow:

ecs-cli compose --file <generated docker-compose file> service up

This will deploy your docker containers on to the default cluster. To check your containers status simply call.

ecs-cli ps

If everything is right, your service should be up and running. Check your EC2 IP address and try access them. Kindly note that you might also need to configure your security group for your EC2 instances as well if you use any other ports than 80.

Hope this helps.