Don’t Repeat Yourself with Anchors, Aliases and Extensions in Docker Compose Files

King Chung Huang
Oct 14, 2017 · 6 min read

Docker Compose files are a great way to define multiple containers and services that work together as a stack. But, it can be easy to end up with repeated blocks in stacks with lots of similar services or configurations. Don’t Repeat Yourself (DRY) by leveraging YAML aliases and anchors, and the upcoming extension fields feature in the Docker Compose file format.

What’s the Problem?

Docker Compose files list zero or more services that are deployed together. A simple example might consist of a web server, app server, and database for a generic application.

version: '3.3'
services:
web:
image: nginx
port:
- 80:80
app:
image: my-app
environment:
- DB_HOST=db
db:
image: postgres

Each service in this example is unique and do not share any options. The Docker image for each service is different. As are configuration options like the published port on the service and the environment variable on the service. The services are not related to each other and do not have repeated sections between them.

As your stack grows, you may start to find common configurations that span multiple services, especially in microservice architectures. The example Docker Compose file for OpenFaaS—a serverless framework for Docker & Kubernetes—defines eight demo services for functions that can be called via the FaaS gateway. Here is an excerpt from the stack for the and services.

services:
# Node.js gives OS info about the node (Host)
nodeinfo:
image: functions/nodeinfo:latest
labels:
function: "true"
depends_on:
- gateway
networks:
- functions
environment:
no_proxy: "gateway"
https_proxy: $https_proxy
deploy:
placement:
constraints:
- 'node.platform.os == linux'
# Uses `cat` to echo back response, fastest function to execute.
echoit:
image: functions/alpine:health
labels:
function: "true"
depends_on:
- gateway
networks:
- functions
environment:
fprocess: "cat"
no_proxy: "gateway"
https_proxy: $https_proxy
deploy:
placement:
constraints:
- 'node.platform.os == linux'

The , , , and options are the same between the two services, and in fact identical across 7 of the 8 services in the stack. Repeating these details can be cumbersome and lead to errors. Changes have to be duplicated many times, and mistakes or omissions can happen by accident. In the context of DRY, repeating yourself is known as WET, or “we enjoy typing” (among other interpretations).

YAML Aliases and Anchors to the Rescue

Docker Compose files are written in YAML. Like JSON, YAML lets you describe typical data types and structures such as strings, numbers, booleans, sequences, and mappings (in fact, YAML is a superset of JSON). But, YAML has many extra tricks that take it beyond just plain old data serialization. One of these goodies is Anchors and Aliases.

Anchors and aliases let you identify an item with an anchor in a YAML document, and then refer to that item with an alias later in the same document. Anchors are identified by an character, and aliases by an character. Here is an example showing an item in a list that is identified by an anchor named , that is then refered to later on in the list.

- &flag Apple
- Beachball
- Cartoon
- Duckface
- *flag

When this list is read by a YAML parser, the value identified by the anchor (Apple) will be filled at the alias when the YAML is composed. The literal anchor and alias are discarded so they don’t appear in the final result.

- Apple
- Beachball
- Cartoon
- Duckface
- Apple

Anchors can be used repeatedly by multiple aliases. So, the value Apple can be repeated several times in the example list. Anchor names can also be reused. Aliases will refer to the most recent instance of an anchor.

DRY Docker Compose Services with Anchors and Aliases

Earlier on, we saw that the example Docker Compose file for OpenFaaS repeated several details between its services for functions. Let’s use anchors and aliases to cut down the repetition.

services:
# Node.js gives OS info about the node (Host)
nodeinfo: &function
image: functions/nodeinfo:latest
labels:
function: "true"
depends_on:
- gateway
networks:
- functions
environment:
no_proxy: "gateway"
https_proxy: $https_proxy
deploy:
placement:
constraints:
- 'node.platform.os == linux'
# Uses `cat` to echo back response, fastest function to execute.
echoit:
<<: *function
image: functions/alpine:health
environment:
fprocess: "cat"
no_proxy: "gateway"
https_proxy: $https_proxy

The anchor and alias here look a bit different than the simple list example! In this case, we are not referring to a scalar value (“Apple” in the list). Instead, we want to refer to the service mapping. To do that, the anchor is placed after is declared, and before the first key-value pair in the mapping.

In , the first key-value pair uses as the key. This is a special key that indicates key-values from another mapping should be merged into this mapping. That other mapping is comes from the alias in this case. So the key-values for , , , , , and from are merged into , first. Then we overide the values for and because they differ from . The end result is the same definitions for and as earlier, but without repeating items like and .

Using Extension Fields to Create Base Services

In the previous example, the service was used to provide the base definition for , which then overrode specific key-values like and . This strategy can be problematic if the services only share some details, but differ in all other aspects. Now, instead of having to keep track of repeated items, you end up with a potentially messier problem of keeping track of overrides. Worse yet, the anchored service may set an option that you don’t want set at all in the aliased service. There’s no way to skip a key-value while merging in from another mapping.

Docker Compose file format 3.4 introduces support for extension fields. Any top-level key starting with in a Docker Compose file will be ignored by Docker Compose and the Docker engine. You can use extensions to declare a partial service containing only the shared options. Because they will be ignored, you can cleanly separate out services meant to be used as anchors only, from services that are actually meant to be deployed.

Here are the same services as before, but with a new service fragment that is declared as an extension. Note that is declared at the top-level, not under .

# Basic function options
x-function: &function
labels:
function: "true"
depends_on:
- gateway
networks:
- functions
deploy:
placement:
constraints:
- 'node.platform.os == linux'
services:
# Node.js gives OS info about the node (Host)
nodeinfo:
<<: *function
image: functions/nodeinfo:latest
environment:
no_proxy: "gateway"
https_proxy: $https_proxy
# Uses `cat` to echo back response, fastest function to execute.
echoit:
<<: *function
image: functions/alpine:health
environment:
fprocess: "cat"
no_proxy: "gateway"
https_proxy: $https_proxy

The service isn’t a valid service definition in a Docker Compose file (it’s missing ). But, that doesn’t matter because it’s named starting with , which Docker will treat as an extension and ignore. The and services both merge it in, then set their specific and . Changes to the service will not have side effects on , because no longer merges in .

Extensions also make it easy to use multiple anchors and aliases to compose service definitions. For example, the OpenFaaS demo stack might one day include Windows-based services. Instead of setting the placement constraint for Linux in , separate OS placement extensions can be created and used.

# Basic function options
x-function: &function
labels:
function: "true"
depends_on:
- gateway
networks:
- functions
# Linux placement
x-linux: &linux-only
deploy:
placement:
constraints:
- 'node.platform.os == linux'
# Windows placement
x-windows: &windows-only
deploy:
placement:
constraints:
- 'node.platform.os == windows'
services:
# Node.js gives OS info about the node (Host)
nodeinfo:
<<: *function
<<: *linux-only
image: functions/nodeinfo:latest
environment:
no_proxy: "gateway"
https_proxy: $https_proxy
# Uses `cat` to echo back response, fastest function to execute.
echoit:
<<: *function
<<: *linux-only
image: functions/alpine:health
environment:
fprocess: "cat"
no_proxy: "gateway"
https_proxy: $https_proxy

Now, services like and can easily be set up as FaaS function services for Linux or Windows hosts. Other services can just as easily use the and anchors without taking on the values in the anchor.

Extensions can be used for any part of a Docker Compose file. The concept here for services apply just as equally to networks, volumes, and environments, for example.

Summary

YAML anchors and aliases let you reference and use the same data multiple times within a single YAML document. You can use them to repeat simple scalar values (such as strings) or key-value pairs in mappings. For mappings, the special key lets you merge in an aliased mapping.

Docker Compose file format 3.4 adds support for extension fields: top-level keys starting with that are ignored by Docker Compose and the Docker engine. You can use them to define composable service defintions by creating service fragments, and then mixing them in to create concrete services. Extensions can be used for any part of a Docker Compose file.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store