Simplifying docker-compose operations using Makefile

Make wrapper over Docker Compose

Every few years there is a paradigm shift in the software industry. The concept of microservices is one of them. Although they are not a new concept, it's only recently that its popularity has sky-rocketed.

Large monolithic services are now being replaced by independent and autonomous microservices. A microservice can be thought of as an application that serves a single, very specific purpose. Like RDBMS, Express app or Solr etc.

It is hard to imagine any new software development today without thinking about microservices and that leads us to Docker.

Docker

Docker has become almost an industry standard to develop and deploy these microservices. From the site itself:

Docker The only independent container platform that enables organizations to seamlessly build, share and run any application, anywhere — from hybrid cloud to the edge.

Docker Compose

Docker compose allows to configure multi-container app. You can configure as many docker containers as you want under a single Docker Compose.

With Docker Compose, you use a YAML file to configure your application’s services and how they connect to each other.

Docker Compose is a tool for defining and running multi-container Docker applications

Two containers on a host

GNU Make

The makeprogram is basically an automation tool for building programs and libraries from source code. Generally though, makeis applicable to any process that involves executing arbitrary commands to transform a source command to a target result. We will specifically use the PHONY targets feature of make to control docker-compose commands.

To tell make what to do, we need a file called a makefile

Our makefile will contain native dockerand docker-compose commands to build image, start/stop/restart container, login to the container, tail container logs etc.

A typical use-case

Let us assume a standard web application with the following components.

• Timescaledb (postgres)
• ExpressJs app
• Ping ( just a dummy container )

This app will need 3 docker containers and a docker-compose file over these containers. Now, each of these containers will have different interaction points. For e.g. timescaledb might have db-like interactions:

• Login to the postgres shell
• Import/Export a table
• Take a pg_dump of table/database

Similarly expressjs might have the following app-like interactions:

• tail a log file
• login to the shell to run some command

Interacting with the containers

Once we have the containers linked using Docker Compose, the next steps are to actually interact with them. Docker Compose provides a docker-compose command and a -f option to take a docker-compose.yml file.

Using this switch we can restrict our interactions to only the containers that are there in the docker-compose.yml file.

Let us look at how these interactions happen using the docker-compose commands. Suppose, we want to login to psql shell, the command might look like:

docker-compose -f docker-compose.yml exec timescale psql -Upostgres

The same command, without using docker-compose, but using the docker might look like:

docker exec -it  edp_timescale_1 psql -Upostgres

Note: It is always better to use docker-compose over docker in such cases, since you dont need to remember the container names.

However, if we have a Makefile wrapper, that exposes a simple command and internally calls these commands, it will look like:

make db-shell

It's very clear how concise the command is using the Makefile wrapper!

Working example

Using our typical use-case above, we can create a docker-compose file as follows:

version: '3.3'
services:
    api:
        build: .
        image: mywebimage:0.0.1
        ports:
            - 8080:8080
        volumes:
            - /app/node_modules/
        depends_on:
            - timescale
        command: npm run dev
        networks:
            - webappnetwork
    timescale:
        image: timescale/timescaledb-postgis:latest-pg11
        environment:
          - POSTGRES_USER=postgres
          - POSTGRES_PASSWORD=postgres
        command: ["postgres", "-c", "log_statement=all", "-c", "log_destination=stderr"]
        volumes:
          - ./create_schema.sql:/docker-entrypoint-initdb.d/create_schema.sql
        networks:
           - webappnetwork
    ping:
       image: willfarrell/ping
       environment:
           HOSTNAME: "localhost"
           TIMEOUT: 300
networks:
   webappnetwork:
       driver: bridge

To manage the above docker-compose and interact with its various containers, we create the following makefile

THIS_FILE := $(lastword $(MAKEFILE_LIST))
.PHONY: help build up start down destroy stop restart logs logs-api ps login-timescale login-api db-shell
help:
        make -pRrq  -f $(THIS_FILE) : 2>/dev/null | awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | sort | egrep -v -e '^[^[:alnum:]]' -e '^$@$$'
build:
        docker-compose -f docker-compose.yml build $(c)
up:
        docker-compose -f docker-compose.yml up -d $(c)
start:
        docker-compose -f docker-compose.yml start $(c)
down:
        docker-compose -f docker-compose.yml down $(c)
destroy:
        docker-compose -f docker-compose.yml down -v $(c)
stop:
        docker-compose -f docker-compose.yml stop $(c)
restart:
        docker-compose -f docker-compose.yml stop $(c)
        docker-compose -f docker-compose.yml up -d $(c)
logs:
        docker-compose -f docker-compose.yml logs --tail=100 -f $(c)
logs-api:
        docker-compose -f docker-compose.yml logs --tail=100 -f api
ps:
        docker-compose -f docker-compose.yml ps
login-timescale:
        docker-compose -f docker-compose.yml exec timescale /bin/bash
login-api:
        docker-compose -f docker-compose.yml exec api /bin/bash
db-shell:
        docker-compose -f docker-compose.yml exec timescale psql -Upostgres

Most of the commands run on all containers, however, using the c= option we can limit the command to only one container.

Once the makefile is ready, we can use it in the following ways:

• make help— list all commands available for make

• make build — build the image from Dockerfile. In our example we have used an existing image of timescaledband ping. However, forapi , we want to build locally. This command will do that.

Build docker container

• make start — is used to start all the containers. To start only one container run make start c=timescale

Start timescaledb container

Start ping container

• make login-timescale— is used to log-in to bash session in the timescale container.

Start bash for timescaledb

• make db-shell — is used to log-in to the psql in the timescale container to run sql queries on the database

Start psql for timescaledb

• make stop— is used to stop the container

Stop timescale container

• make down — is used to stop and remove containers. To delete specific container use make down c=timescaledb or make down c=api

Stop and remove all containers

Conclusion

Even though Docker Compose provides comprehensive sets of commands to manage the container, sometimes the commands can become long and hard to remember.

This one trick of Makefile has helped us to quickly and easily interact with the containers under the docker-compose file.

• You only interact with the containers that are part of the Docker Compose for that project. Without knowing about other running containers
• In case you forget the commands, you can do make helpand see all available interaction commands
• No need to remember a long set of arguments to for eg. tail a log file or login to db-shell. docker-compose -f docker-compose.yml exec timescale psql -Upostgres vs make db-shell
• Customize the Makefile to do project-specific things. For eg. quickly take the backup of the DB, run superctl status etc.
• The entire team uses the same set of commands, reducing confusion and errors