Continuous Integration with Docker on Amazon Elastic Beanstalk

  • deploy to single-container Elastic Beanstalk (EB),
  • use Circle as CI engine, I am pretty sure that other services (e.g. Travis, Codeship, etc.) would offer equivalent functionalities,
  • use a private registry to store compiled Docker images: I use Quay.io because it is cheaper than Docker Hub and works just fine for what I need,
  • deploy a Python Django application, however this is not strictly necessary as most of the steps apply to any kind of web application.

Step 1. Elastic Beanstalk on AWS

[profile eb-admin]
aws_access_key_id = ACCESSKEY
aws_secret_access_key = SECRETKEY
  • Docker environment,
  • set it to auto-scale,
  • do NOT use RDS that comes with EB, it is more flexible to setup your own RDS and hook it to EB later,
  • sat yes to VPC,
  • set some other reasonable settings, assuming you have basic knowledge of AWS to handle security groups, key-pairs, etc.
mkvirtualenv aws
# probably not strictly necessary, but will come useful later…
pip install awscli
pip install awsebcli
eb init --region eu-west-1 --profile eb-admin
  • it should show the EB applications you have created in your AWS account, select the one you want to use from the list, it should match the one just created with AWS Web Console,
  • all other options should be automagically detected from the online settings.
  • skip platform auto detection and choose Docker (if a Dockerfile is found, it probably will not ask for platform details)
  • choose an existing keypair ssh configuration
!.elasticbeanstalk/config.yml
  • instance role (or EC2 role) is used by EB to deploy and have access to other resources
  • service role is used by EB to monitor running instances and send notifications

Step 2. Docker

  • on Quay.io create a robot account and assign write credentials to the private repo
  • download the .dockercfg file associated to the robot account and put it BOTH in ~ on your local machine (so you can authenticate on Quay.io from the command line) and in your repository’s root (will be used later by Circle)
EXPOSE 80
CMD [“your-startup-file.sh”]
docker push quay.io/my_account/my_image:latest

Step 3. Circle CI

machine:
python:
version: 2.7.3
services:
— docker
dependencies:
cache_directories:
— “~/docker”
pre:
# used to run deploy script
— pip install awscli
# used to authenticate on quay.io
— cp .dockercfg ~
override:
# cache your TESTING docker image on circle CI
— if [[ -e ~/docker/image.tar ]]; then docker load -i ~/docker/image.tar; fi
— docker build -f Dockerfile.test -t test ./project_root/
— mkdir -p ~/docker; docker save test > ~/docker/image.tar
test:
post:
# run tests
— docker run test python runtests.py
deployment:
staging:
branch: release/stg
commands:
# this script might be more complicated
# with a few more arguments maybe
— ./deploy.sh $CIRCLE_SHA1 eb-stg-env

Step 4. Deploy

  1. build docker image and push to Quay.io
  2. create a Dockerrun file for EB (read details below) and push it to S3
  3. tell EB to create a new application version and deploy it
# deploy.sh
#!/bin/bash
# name of the deploy image on quay.io
BUILD=”quay.io/my_account/my_image:$SHA1"
# store quay.io authentication to S3
aws s3 cp .dockercfg s3://$EB_BUCKET/dockercfg
# deploy tag
SHA1=$1
# elastic beanstalk environment to deploy
EB_ENV=$2
APP_NAME=”MyEBApp”# remember the name of the bucket I asked to take note
# above in this post? here is where you use it!
EB_BUCKET=”bucket-where-EB-deploys-its-stuff”
# where to put things in the S3 bucket
PREFIX=”deploy/$SHA1"
# build main image
docker build -f Dockerfile.build -t $BUILD:$SHA1 .
docker push $BUILD:$SHA1
# replace vars in the DOCKERRUN_FILE
# so that EB knows where to pick things from
DOCKERRUN_FILE=”Dockerrun.aws.json”
cat “$DOCKERRUN_FILE.template” \
| sed ‘s|<BUCKET>|’$EB_BUCKET’|g’ \
| sed ‘s|<IMAGE>|’$BUILD’|g’ \
| sed ‘s|<TAG>|’$SHA1'|g’ \
> $DOCKERRUN_FILE
aws s3 cp $DOCKERRUN_FILE_WEB s3://$EB_BUCKET/$PREFIX/$DOCKERRUN_FILE
rm $DOCKERRUN_FILE
# Create application version from Dockerrun file
echo “creating new Elastic Beanstalk version”
aws elasticbeanstalk create-application-version \
--application-name $APP_NAME \
--version-label $SHA1 \
--source-bundle S3Bucket=$EB_BUCKET,S3Key=$PREFIX/$DOCKERRUN_FILE
# Update Elastic Beanstalk environment to new version
echo “updating Elastic Beanstalk environment”
aws elasticbeanstalk update-environment \
--environment-name $EB_ENV \
--version-label $SHA1
{
“AWSEBDockerrunVersion”: “1”,
“Authentication”: {
“Bucket”: “<BUCKET>”,
“Key”: “dockercfg”
},
“Image”: {
“Name”: “<IMAGE>:<TAG>”,
“Update”: “true”
},
“Ports”: [{
“ContainerPort”: “80”
}],
“Logging”: “/var/eb_log”
}

Step 5. Tweaks, env variables, etc.

[
{
“Namespace”: “aws:elasticbeanstalk:application:environment”,
“OptionName”: “DJANGO_SETTINGS_MODULE”,
“Value”: “$DJANGO_SETTINGS_MODULE”
},
{
“Namespace”: “aws:elasticbeanstalk:application:environment”,
“OptionName”: “SECRET_KEY”,
“Value”: “$SECRET_KEY”
}
]
# update-env.sh
#!/bin/bash
OPTIONS_FILE=”EB-options-$(date +%Y-%m-%d:%H:%M:%S).txt”
cat EB-options.txt.template | envsubst > $OPTIONS_FILE
aws elasticbeanstalk update-environment \
--environment-name $EB_ENV \
--option-settings file://$OPTIONS_FILE
rm $OPTIONS_FILE

The end!

--

--

specialization is for insects

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store