Diy ECS — All the cluster without the Cloudformation

I had trouble finding a good tutorial for ECS that didn’t use the quickstart or the CLI. I’m not the biggest fan of Cloudformation and I find it generates a lot of mess and makes it harder to really tweak your AWS environment.

We’re going to set this up properly so no cutting corners, we’re creating an Autoscale group and an ELB as well.

So here’s my adventures in manual ECS from scratch.

Then most importantly, learn the value of the cancel, we’re forging our own path here. So hit cancel and we’ll begin.

Step 1: Cluster all the things

There, all done! Well, not quite!

Step 2: Setup the IAM Roles

We need to create two roles, one role for the EC2 instances (we’re calling this “staging”). This tutorial assumes you will be using a private docker repository but if you’re not, feel free to ignore the S3 steps. I’d still recommend doing it as a best practice, it’s more flexible to manage the ECS config.

Unfortunately, the IAM console is trying to be more ‘helpful’ and less diy. But we can skip the steps by selecting “Amazon EC2 Service Role” and then next step to leave the policies blank. Once the role is created, expand the inline policies and create one.

Select a custom policy and then setup two policies, one for ECS and one for your S3 bucket. Name it whatever makes sense (“staging-ecs and staging-s3”)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ecs:CreateCluster",
        "ecs:DeregisterContainerInstance",
        "ecs:DiscoverPollEndpoint",
        "ecs:Poll",
        "ecs:RegisterContainerInstance",
        "ecs:Submit*"
      ],
      "Resource": "*"
    }
  ]
}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::your-deploy",
                "arn:aws:s3:::your-deploy/*"
            ]
        }
    ]
}

Step 3: Add your ecs.config to your S3 bucket

Create a file s3://your-deploy/ecs.config in your S3 bucket with the following data.

ECS_CLUSTER=test
ECS_ENGINE_AUTH_TYPE=dockercfg
ECS_ENGINE_AUTH_DATA={“https://index.docker.io/v1/":{"auth": “get_this_from_your_~/.docker/config.json”,”email”: “email@dockerhub.com”}}

Step 4: Create your Autoscale group to be in the cluster

Create a launch configuration with the ecs-optimized AMI if you don’t want to install the ecs client manually which is the AMI named amzn-ami-2015.09.a-amazon-ecs-optimized

Remember to set the Iam Role to “staging” and then under advanced details you want to set the user data to

#!/bin/bash
yum install -y aws-cli
aws s3 cp s3://your-deploy/ecs.config /etc/ecs/ecs.config

Create your autoscale group, select the launch config you just created, don’t attach an ELB.

Step 5: Definition is important

Back in ECS, head to Task Definitions and ‘Create a new Task Definition’

Define your task and container definition, most of the details are fairly well explained. *note please use your own container name

Step 6: Balance your Load

Create a load-balancer with the port forwarding setup to go form http to the port mapping you defined above (in this case 8080). Make sure the security group is set to allow traffic from the Elb to the instance.

Step 7: Choose your own Service

In ECS still, go to Clusters and click on your existing cluster, “Staging.” We want to create a new service to run our staging task.

Fill in the existing settings and press create IAM role to setup the base ecsServiceRole.

Step 8: Sit Back and let ECS Dance

That’s it, ECS will now make sure 1 task of your docker image is running at all times. No cloudformation, complete control over your autoscale group and services.

Footnote

Links to documentation at Amazon

Amazon ECS Container Instance IAM Role

Amazon ECS Service Scheduler IAM Role

http://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_container_instance.html

http://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html