My ideal docker deployment and orchestration tool

I should probably start by pointing out that to my knowledge this tool doesn’t exist yet and while I would definitely use this, I’m not capable of creating it on my own.

EDIT 20th April — I’ve started an org on github, where you will find a refined specification for Airdock. This post contains ideas that have since been refined, changed or removed. I shall hopefully commence development soon.

There are many docker deployment tools in development, so maybe this idea is premature, but I believe there are some key differences in the way this works that I prefer.

I’ve decided to call this imaginary tool AirDock and here is how I think it should work.

AirDock is in three parts. The first part is a service which runs inside a Docker container on all of the users web hosts. This “service” exposes an API which can be used to to deploy applications and other services within containers. Etcd is used here for service discovery, as well as linking hosts, storing formation settings and useful metrics.

The second part is a CLI which talks to the AirDock API to show details and make deployments. Finally, AirDock provides a web app with a formation overview including metrics and other gems.

The tool does not require applications to be written in pre determined languages as it uses dockerfiles to boot apps and services with the correct libraries and dependencies. Airdockfiles may also be required when deploying, as these provide further information on how the deployment should occur across the formation.

Formations

Formations are a group of hosts that work together to load balance and ensure reliability of the users entire stack. A formation can exist on the cloud, or on a local dev machine and can be made up by several hosts, or just one. Access to the formation API is through any of the hosts, but of course this can be restricted.

Hosts can be assigned custom smart tags which allow the user to deploy specific services, to specific hosts. For example, we could deploy a blog to the tag “blogs”, meaning the blog will be deployed to any hosts assigned with that tag. Any hosts with the tag “blogs” added to the formation at a later date, will be given the choice of having the blog deployed there also.

Services and Applications

Along with the Dockerfile, services and application should have an airdockfile containing URI’s to setup, services that it exposes and those that it requires, as well as test-suites to run etc.

Services such as databases each have their own unique way of scaling horizontally, which provides a challenge when trying to automate the process. AirDock would simplify this with service build packs which could be made publicly available.

Users can deploy load balancers of their choosing and these can be hooked into etcd to trigger restarts and vhost changes when deployments are made across the formation.

When deploying changes, app source code can be sent down the line as a tarball, or users can make use of github hooks to deploy on a release. New containers will spin up every time a deployment is made with the newly deployed code in place. Test suites can be run before going live ensuring the setup is working as expected.

On a personal level, I’ve chosen not to use git push to deploy because I would like to be able deploy and run tests inside my dev formation without having to commit every change. Git also requires all files to be checked in and managing dependencies in git can get tedious. However, git functionality could always be added later.

Other things to note.

1. Users can redeploy an entire formation anywhere, even to a local machine so apps can be tested in an environment almost identical to production.
2. Formations play nice with several applications, so that developers can build small modular API’s and web apps.
3. Upon deploys or adding hosts, other applications may need to re-deploy in order to get the latest configurations.

An example CLI

Here we have an example command line interface for AirDock. It provides a simple way to manage the formation, including deployment of services and applications.

Logging into a formation

This logs you in to an existing AirDock formation, in this case the “my-massive-stack” formation. If this is the first time running this command, the CLI will request an IP address of an airdock host within that formation.

$ airdock my-massive-stack
user: foo
pass: ****************

Showing details about the formation

Once logged in, you can run commands in the context of that formation. Here we would be able to see an overview of the entire formation and drill down to more specific information if we wanted.

$ airdock show
 
IP          CONTAINERS                         TAGS
10.0.0.1    etcd, airdock, loadbalancer1,      main
10.0.0.2    etcd, airdock, www1, mongo1        web, dbs
10.0.0.3    etcd, airdock, www2, mongo2        web, dbs
10.0.0.4    etcd, airdock, www3, mongo3        web, dbs

Deploying an App

Assuming both a dockerfile and airdock file are present in the current directory along with any app source code, the following command deploys and scales that app to any hosts with the tag named “web”

$ airdock deploy --scale web

You can also specify how many hosts tagged with “web” you want it deployed to. In this case, its only to one host.

$ airdock deploy --scale web:1

Or specify multiple tags such as “web”, “frontend” and how many of those to deploy to.

$ airdock deploy --scale web:frontend:1 

Deploying a service

The following command looks for a mongodb build pack, and follows the instructions. The —name flag allows you to specify a unique handle for the newly formed service and this name will be passed to the rest of formation. The —scale flag allows you to specify tags like above to deploy to.

$ airdock service mongodb --name mongo --scale dbs:3

Well there you have it, a basic outline of my ideal docker deployment and orchestration tool. If you have any thoughts, please get in touch.