Docker-compose.yml: from V1 to V2

A short guide for converting to the new format

Lately (early february 2016) Docker officially announced the availability of the new version (V2) of the docker-compose.yml file format, with support for the latest features in docker engine: volumes and networks. I wanted to convert my existing docker-compose files, and what happened is that I lost some time looking around or looking for help (that I got :) ). Here is my sharing of some of the info I gathered, just in case you would need it…

I assume that you have already knowledge of Docker and of docker-compose V1 files

A docker-compose V1 file

So, I had something like this on a docker-compose V1 file:

dbpostgres:
 image: postgres:9.4 #define the image to get
 volumes_from:
  — dbstore
 ports:
  — “5432:5432”
express-app-container:
 build: .
 ports:
  — “3000:3000”
 volumes:
  — ./:/app
 links:
  — dbpostgres

this defines a small development environment built of the 3 canonical pieces:

• a data store : a docker data-only volume container named: dbstore
• a postgres db container named: dbpostgres
• a node express application container named: express-app-container

It uses an already existing data-only container (dbstore), created with a command like this:

docker create — name=dbstore -v /var/lib/postgresql/data postgres:9.4

The links option at the end of the file will instruct docker-compose to open a network between the express-app-container and the dbpostgres linked containers. Docker-compose will also change the etc/hosts file on the express-app-container container, adding an host name (dbpostgres), so the express.js application can reach the postgres database in the linked container with something like this:

var pg = require(‘pg’);
var connectionString = “postgres://postgres:postpwd@dbpostgres/mybigdb”

This kind of docker-compose.yml file was working pretty well, but I wanted to move to a V2 file, just to taste how the freshly announced new features of the V2 could help.
I started a step by step move (never change many things at once, else how to discover wich one is wrong?) toward the new format, and here are the recordings of the steps.

Step one on v2 — only minimal changes

here is the first version:

version: ‘2’
services:
 dbpostgres:
  image: postgres:9.4 #define the image to get
  volumes_from:
   — container:dbstore #this is a container already defined
  ports:
   — “5432:5432”
 express-app-container:
  build: .
  image: express-app #assign a name to the built image
  ports:
   — “3000:3000”
  volumes:
   — ./:/app
  links:
   — dbpostgres

like you can see, just minimal changes:

version: ‘2’

this is mandatory for version 2; if missing, the file will be considered of version 1.

services:

Because now we can have three top level objects (services, volumes and networks ), we should define the one we are adding options to. For V1 there was only services, so no need to specify.

volumes_from:
   — container:dbstore #this is a container already defined

in the volumes_from we now should add a specification (container) if (like in our case) the volume we want to use is a container (a data volume container). This is because docker-compose v2 is also supporting the new concept of volumes, introduced lately in docker engine. If we don’t add the specification, using only the container name, as in:

volumes_from:
  — dbstore

docker-compose will be unable to find the container and will raise an error.
Well, this is the first version of the file, everything as in V1, just new syntax.

Step two — removing links

Now we wanna start using some of the new features of V2 and, as first, we can remove the links option. When docker-compose execute V2 files, it will automatically build a network between all of the containers defined in the file, and every container will be immediately able to refer to the others just using the names defined in the docker-compose.yml file.
So we don’t need links anymore; links was used to start a network communication between our db container and our web-server container, but this is already done by docker-compose. Here is the new version of our file:

version: ‘2’
services:
 dbpostgres:
 image: postgres:9.4 #define the image to get
 volumes_from:
 — container:dbstore #this is a container already defined
 ports:
 — “5432:5432”
 express-app-container:
 build: .
 image: express-app #assign a name to the built image
 depends_on:
 — dbpostgres #wait for dbpostgres to be started, not for ready
 ports:
 — “3000:3000”
 volumes:
 — ./:/app

we removed the lines:

links:
 — dbpostgres:postgresdb

but added:

depends_on:
 — dbpostgres #wait for dbpostgres to be started, not for ready

The links option had 2 different meanings: one was defining a network, the other was defining an implicit dependence between the linking and the linked. This was instructing docker-compose to first start the creation of the linked service, and then the the creation of the linking one. Without links:, the latter is lost, so depends_on does the missed effect, telling docker-compose of the dependency, but using a more explicit declaration.

Step three — using volumes:

Now, we can use still another functionality of V2, the high level volumes object, specific for defining the creation and naming of volumes (volumes are a recent addition of docker engine), wich can then be used by the services defined into the same docker-compose.yml file. Volumes are (almost) substituting the data-only containers.
Here is the new version of the file:

version: ‘2’
services:
 dbpostgres:
  image: postgres:9.4 #define the image to get
  volumes:
   — pgvolume:/var/lib/postgresql/data #using the declared volume
  ports:
   — “5432:5432”
  express-app-container:
  build: .
  image: express-app #assign a name to the built image
  depends_on:
   — dbpostgres #wait for dbpostgres to be started, not for ready
  ports:
   — “3000:3000”
  volumes:
   — ./:/app
volumes:
 pgvolume:  #declare a volume named pgvolume

we added the new section:

volumes:
 pgvolume: #declare a volume named pgvolume

that defines a new volume named pgvolume, located in the default place, that is local. But the new section allows also to specify different drivers (see the docker documentation Volume configuration reference), allowing the definition of volumes residing outside of local, for example on the cloud.
In addition, because docker-compose recognise volumes, it’s also safe to execute the command:

docker-compose rm 

to remove all of the containers defined in the docker-compose.yml file. Docker-compose will not remove volumes, saving the data inside and making them available for being reused.
The other change in the file is:

volumes:
   — pgvolume:/var/lib/postgresql/data #using the declared volume

we removed the volumes_from option and added the volumes one, used to point to the pgvolume defined in the volumes object.

Final step: using external volumes

Once we have a volume defined, we not only can use it in the file where it’s defined, but also on other docker-compose.yml, just defining it as external (se effectively reusing it):

version: ‘2’
services:
 dbpostgres:
  image: postgres:9.4 #define the image to get
  volumes:
   — pgvolume:/var/lib/postgresql/data #using the declared volume
  ports:
   — “5432:5432”
  express-app-container:
  build: .
  image: express-app #assign a name to the built image
  depends_on:
   — dbpostgres #wait for dbpostgres to be started, not for ready
  ports:
   — “3000:3000”
  volumes:
   — ./:/app
volumes:
 pgvolume:
  external:
   name: nodedocker_pgvolume

Here the changes are only on the volumes section.
With external we say docker-compose to look for a volume not defined in the same docker-compose.yml file, but looking for it on the docker environement.
The volume to look for is named nodedocker_pgvolume, this is the name that docker-compose assigned (in our case) to our pgvolume the run before. Docker-compose add namespaces to keep projects separated, and the namespace name, if not superseded, is taken from the name of the directory holding the docker-compose.yml file (nededocker in our case).

We still have pgvolume as the name to be used internally to the docker-compose file, so our reference in:

volumes:
   — pgvolume:/var/lib/postgresql/data #using the declared volume

doesn’t need changes. Obviously we should have already created the volume…

Well, this is the V2 version of our initial V1 docker-compose.yml file, just what we wanted.
Like you can see, the latest version is adding a set of new features, but also some duplication, in the sense of different supported way to specify (almost) the same thing. This is due to compatibility reason and to the fast pace all of the docker echosystem is moving on; new features supersede old ones, but the old ones require times to be cleaned off.

Final though

That’s all, folks, I hope this could make life a bit easier for someone, and if you have suggestions or you see errors, feel welcome to correct me, learning is a never ending activity, lukily.