Taking Control of Container Placement
How to Use Tags and Constraints in ContainerShip for Fine Grained Control of Where Containers Run on Your Cluster
Services like Amazon ECS let you run Docker containers on a server cluster, but ECS gives you no control over how or where those containers are placed across the cluster.
Why does placement matter?
Application Specific Hardware Configuration
Forcing certain applications to live on servers with a special hardware configuration is often a requirement.
One example is a large group of servers with tons of RAM you would like to run Memcached on. You need to be able to force your Memcache application to only launch containers on these specific servers.
Another example is the case where you have a SAN or a large amount of local storage on a group of servers that you are storing user uploaded content on. You need to be able to ensure that your service gets deployed to the right machines to make use of the hardware.
With other solutions this is not possible or requires you to jump through hoops, but ContainerShip makes it easy.
Equal Distribution Across Multiple Data Centers
Often it is desirable to be able to evenly distribute the containers for an application across multiple data centers for high availability. This is important for various reasons:
- When load balancing you want to send traffic to containers that live in the same data center and avoid cross-zone data transfer charges.
- If one of the data centers goes down or has a failure, you want to ensure that you have containers for each application in another healthy data center.
Apps That Need One Container On Every Follower
When you’re running everything in your infrastructure as a container, there is going to come a point where you will need to launch something that needs to run on every server in the cluster.
- Logging agents: Rsyslogd, Logstash forwarder, DataDog
- Monitoring agents: Sensu, nagios-nrpe
- Metrics collection: Collectd, cAdvisor, New Relic
When you are not able to do this, it forces you to run these tools outside of containers, or manage them in a completely different way than the rest of your apps. They become another one-off custom deployment that you need to automate yourself with configuration management, and properly communicate about and train your team on.
ContainerShip Tags System
The ContainerShip tags system makes it easy to force placement of an application’s containers based on metadata associated with your follower hosts.
The ContainerShip cloud-hints plugin automatically adds some useful tags to your follower hosts, like the provider they are running in, and which region or data center. The image above shows the tags that were automatically added on a cluster running in Amazon Web Services.
Utilizing Tags with Applications
As an example, we could force an application’s containers to only be placed in us-east-1d by setting the same tag in our application.
cloud.availability_zone=dBelow you can see example of how this would look in the web interface.
Setting this tag and saving it will force our redis application’s containers to only be deployed to follower that are in availability_zone d.
Setting Your Own Tags
You can set your own tags on hosts and applications to get the desired placement and distribution that you require. Tags can be set via the web ui or the API.
To set a tag for an application via the API you can do the following:
curl -X PUT -H ‘Content-Type: application/json’ http://leader.host.ip:8080/v1/applications/redis -d’{ “tags”: { “mycooltagname”: “value” } }’To set a tag for a host via the API you can do the following:
curl -X PUT -H ‘Content-Type: application/json’ http://leader.host.ip:8080/v1/hosts/b86b923419594c5d -d’{ “tags”: { “mycooltagname”: “value” } }’Constraints
Constraints are specialized tags that perform some extra functionality automatically. Read on for examples of the available constraints in ContainerShip.
Per Host
If you wanted to run one container for an application on every follower host, perhaps to run your sensu agent on every follower, you could use the per_host constraint in your application definition.
You can see how the per_host constraint is set in the application definition within the web ui below.
In this case we have set the constraint to launch 1 container for our containership-haproxy application on each follower host in our cluster. We could set the value to any number we wanted though to scale up further. For example if we wanted to run 4 containers of containership-haproxy on each follower, we would simply change the value in the tag to 4.
More Constraints Coming Soon
We are currently in the process of adding additional constraints to the platform such as:
Exclusive — Run a maximum of 1 container for an application on every follower host.
Partition — Split containers equally across the available hosts for a given tag.
Conclusion
As you can see, having access to constraints and being able to properly place your containers across your server fleet with tags is vital to being able to scale in a sane way that makes the best use of your hardware resources. Check back in to our blog soon and sign up for our mailing list to get further updates and announcements about constraints and ContainerShip in general.
Further information is available at our official documentation site.
