Ephemeral Development Environments
Written by Michael Hume on May 5, 2018
At GumGum we build alot of in-house web products. The development for these products follow a dev/stage/prod pipeline, but often times a single development environment is not enough. This post describes our migration to using traefik and drone to build out dynamically ephemeral preview development environments for our web products.
We run a typical dev/stage/prod pipeline for most our web projects. These ran under the convention of `<env>.<project>.domain` and have usually been a place to showcase an upcoming feature that needed review by our product team. But there came a need to review multiple coming features being developed in tandem for a single project.
We have been using jenkins as a point where developers could select what branch they wanted to deploy to dev, but we wanted to change this so developers could just focus on commiting code. We’ll later use that as the trigger for deployments. We would also need to create some dynamic routing in the process of deployments to match a url to a version of the application.
Two tools we landed on to solve this problems are Traefik and Drone. Traefik is a reverse proxy load balancer that is aimed at Docker and microservices. It requires almost zero configuration. Drone is Continuous Delivery platform that is built on container technology and has replaced many of our jenkins jobs. Rather than managing build and deploy steps via groovy in a Jenkinsfile, Drone uses a .drone.yml file that closely resembles the docker-compose.yml specifications.
The web team uses git-flow in their development. This results in branch names such as:
We can use these branch names for generating the urls to linking to dev builds. This’ll also make it helpful when one might want context of what they are looking at. Preview urls for projects would look like so:
We are using the branch name as a subpath in the url, though this might not work in all use cases. Another option would be to use the branch name to create a subdomain, but that would also require a rewrite of the branch to make it compliant for urls.
Drone handles the two steps; building and deploying; for the pipeline of our application. Drone plugs into popular git hosting such as gitlab, bitbucket, and github. It works around the concept that webhooks will trigger a job. To get started we add a `.drone.yml` to the root of a project. We then define build steps in the pipeline of the drone.yml. In the example below we define three steps. `build_node_image` `deploy_stage` and `deploy_prod`. Drone uses containers for every build step. The idea being you bring your tools and workspace using different containers at each step.
Now when a developer commits to the project, a webhook will trigger the drone pipeline and run certain steps when conditions are met. The first step `build_node_image` will search for a `Dockerfile` in the repo; build that image; and push it to the appropriate docker registry. The deploy steps will deploy to the appropriate containers to their cluster depending which branch had been committed to. You might notice in the `.drone.yml` file is we are defining the `$family` variable in the `commands` section and not the `environment` section. This is because drone is not able to expand environment variables in the environment section.
To deploy to ecs we use ecs-cli and use a templated docker-compose.drone.yml file to generate the task definition to deploy to the ecs cluster. Though it is easy to see how this could be adapted to a difference orchestrator (nomad/swarm/k8s). Drone also makes available a slew of environment variables to use during the build. We are using `${DRONE_REPO_NAME}` and `${DRONE_BRANCH}` in both the ecs-cli deploy command as well as for labels that Traefik will use to generate urls.
Below is an example of the docker-compose.drone.yml we use to deploy with:
Traefik is launched at boot with the ` — ecs` flag, that tells Traefik to look out for an ecs-agent. Both Traefik and the ecs-agent run as containers on each ecs instance in the cluster. With Traefik running along with the ecs-agent, all we need to do now is append the appropriate label tags to the `docker-compose.drone.yml` we deploy with. Traefik has a robust number of frontend matchers and preprocessors. We’re just interested in matching a hostname and path; then stripping that path before forwarding the request onto the container running inside the cluster. Traefik continuously polls ECS for a list of containers and reads the labels on those containers in order to generate the appropriate load balancing/routing rules.
The resulting architecture:
This post primarily focused on the build and deploy to create a url makes the dev branch retrievable. We haven’t discussed how to setup autoscaling your cluster, now that building out dev branches is just so simple There are a ton of ways to solve this same problem. What I like about this solution with Traefik and Drone is that it is very simple to drop into any architecture one might be running.
They also both have very slick ui’s for visibility:
Traefik requires almost no config and works with just about anything you can throw at it. We are not running service discovery agent and regenerating templates on the fly. Or we are, but Traefik is handling it all.
Drone has sped up our development process now that deployment is not a manual step and is rather triggered by git commits. This also helps us practice better git branch and commit techniques to enforce code promotion and testing. And with drone, docker is a first class citizen. All build steps using docker containers as the plugin that will provide whatever behavior is necessary for that build step.
