From 0 to Fargate — Migrating Your App to Docker
Over the course of the past few months, the TryNow team at Quarkworks has been migrating the entire TryNow application stack to run on docker with full CI deployments for development and production to AWS ECS Fargate. We’ve learned a lot through the process and I wanted to write this series to help you go from 0 to Fargate in production.
With TryNow, we were able to migrate our development environment to running on ECS completely in the span of a month or two. After ensuring stability for a month or two, we recently migrated our production environment to ECS in about 30 minutes with zero downtime. Leading up to the migration process involved making changes to our app to closer align with the 12-factor app methodology. While every application has different requirements and caveats, the core changes we had to make are outlined below:
- Dependencies — Explicitly define dependencies
- Configuration — Store configuration in the environment (not the code or container)
- Processes — Execute the app as one or more stateless processes
- Environment parity — Keep development, and production as similar as possible
- Logs — Treat logs as event streams
- Admin Process — Run admin/management (ex. migrations) tasks as one-off processes
Starting the migration process with your existing infrastructure can be tough. This guide will break the process into bite-sized chunks that you can take to carefully and strategically migrate your production application to docker on AWS ECS Fargate. An important concept to remember is to make every change backward compatible, so that the existing apps do not break. It’s tempting to make all the changes at once but that inevitably leads to frustration and complications if the application is under active development and already running in production. The following sections will outline the steps you can take to Dockerize your app.
Dependencies
This step should be fairly simple for most apps since this is built into most modern frameworks now. To have a reliable docker build, your application must have explicitly defined dependencies. How that looks will vary depending on your application but common methods include: requirements.txt, package.json, gemfile, etc.
Make sure to explicitly define a versioning strategy you feel comfortable with. Make sure your dependencies don’t use “latest” as that will allow them to take any major updates when they are released, which can break your application. You don’t want surprises in production!
Dockerfile
You will need to create a production-grade Dockerfile for each one of your apps. Our stack includes 2 Node.js apps, a Ruby on Rails app, and a Django Python application, each of which needs a separate Dockerfile. I recommend starting with the base image for the language (explicitly versioned of course). There are lots of pre-built images with extra goodies that appear to make it easier at first. The lack of fine-grained control of what is in your container and how it’s running can be detrimental to creating a reliable, robust, and ultimately rock-solid production environment.
When creating your Dockerfile, try to have only one Dockerfile for each environment including local development. This is not always possible, for example, our node.js app has a package step that requires certain env variables during a build. The Django and Ruby apps each have only 1 Dockerfile which makes it easier to maintain and ensures that each build in each environment is exactly the same. Make sure that you do not use a
“for development only” server in your Dockerfile (ex. npm run start or django-admin runserver ). These servers are not designed to handle production workloads and should not be used when creating your production-grade Dockerfile.
Processes
It is fairly common for backend applications to have multiple different tasks that they need to perform. For example, our Ruby application has cron tasks, SQS queue workers, and a web API. While you could roll all of those functions into a single running container, a better approach is to separate them out into three separate containers. This doesn’t mean you need three separate Dockerfiles. You can design the Dockerfile so that it can handle all three of those tasks.
When the container starts, a startup script will determine which of those functions the container will be performing by initializing the appropriate processes. This has many advantages in that it decouples your processes and ensures that high CPU or memory consumption on your batch jobs will not affect your API’s responsiveness. Additionally, if for some reason a batch process crashes you do not have the potential of bringing down your web API with it.
Environment Parity
Having your development and production (and any other additional) environments mirror each other as close as possible is critical to catching errors before they happen in production. Any difference between your environments opens the door for issues that only occur and conversely can only be fixed on production. Using AWS ECS makes this easy because your clusters will be set up in the exact same way so your environments should be very similar. The following article will cover setting up AWS ECS.
Environmental Variables
Your application environmental variables will need to be centralized and not hard-coded. Your application configurations should be loaded at the start of your container or application. There are many approaches to this. Some choose to store their application configurations in the database. On TryNow we choose to use the AWS SSM Parameter store. Our credentials for development and production are stored in the Parameter Store and pulled by application and environment on runtime.
Since our applications were already running in production, the parameter store was added in a way that does not break backward compatibility with the current deployment system during the transition. For the Ruby application, we load environmental variables with Figaro when the application loads. To leave this system intact, we created a bash script that would generate the configuration file with the values from the Parameter Store when the container starts before the Ruby app is loaded. For the script to access the Parameter Store the container has to be provided with AWS credentials as environmental variables. This will be discussed in the following article on AWS and CI setup for deployment.
Health Check
Each application container that is connected to an application load balancer with a public URL(ex. APIs) will need to have a health check endpoint. How to build this into the application will depend on the language and framework. There are different plugins and add-ons for most languages to make this easy. If for some reason, you are unable to add an explicit health check to the application. You can also configure the load balancer health check to use an endpoint that will return a valid (non-500s) HTTP response code.
