Advanced Docker Compose template for Golang

At Raidboss.io we using Golang as the main language for the backend microservices. It would not be a surprise that we wrap our services into Docker containers with Docker Compose.

At the very beginning of our first application, a lot of questions have arisen. What Dockerfile to pick? How to organize Docker Compose files? What is an efficient way to run containers and build Golang apps?
Our development flow is not unique, but it appears there is no suitable and ready-to-use answer to all of them in one place.

Goals behind the template

Let’s make a quick overview of why any of the existing templates for a Docker + Docker Compose + Golang triad do not fit us.

We want to:

• Be able to debug Golang apps (via Goland IDE in our case)
• Keep a production container and image as minimal as possible
• Be sure that development and debug tools are limited to the development environment only
• Use the single Dockerfile for all environments (development, staging, production)
• Reuse our Dockerfile across different Golang apps
• Use the same docker-compose file for regular development and debugging mode
• Keep a development experience as smooth, fast, and effortless as possible

There is nothing extraordinary here, isn’t it?

What is wrong with the existing solutions? If you start to google a bit about the recommended approach you quickly find a magic term from the Docker ecosystem. This term is — multi-stage.

As we have an `advanced` prefix in the title we will not go into details of what is it and how does it work. If you are looking for the advanced solution you know the basics.

But what is new in our approach that we are using three stages instead of two as it is common to see.
Below you will find all the details of our structure and precise explanation of each string.

Template

Disclaimer: in terms of this template we do not showcase or recommend the structure of filenames and folders. The goal of the template is purely related to the contents of Dockerfile, docker-compose, and approach to run/build your code. You can apply the same ideas to the folders structure of your choice. In later plans, there is a separate post with the internals of our structure.

If you do not like to read explanations and prefer to explore everything yourself, take a look at the repository on the Github straight away.

But if you are still reading, let’s dive into the details. Also note, that the local Golang installation is required (1.11+) because the default examples built with Go Modules. Goland version 2019.2 is recommended.

In the repository you will find the next files (in addition to a couple of auxiliary files like .gitignore and readme):

├── app
│   ├── go.mod
│   └── main.go
├── Dockerfile
├── Makefile
├── docker-compose-dev.yml
└── docker-compose-prod.yml

As you can see the structure is pretty simple.
The source code is in the `app` folder.
Using `docker-compose-dev.yml` (for the local usage) and `docker-compose-prod.yml` (for the production usage) we prepare a Docker image from the Dockerfile and build/run our application.

Three stages

Why are we using three stages instead of two?
We think about each stage as an atomic task with a particular result at the end. If you take a look at the Dockerfile you find three descriptive stages.

`env` stage

The result of the `env` task is an environment, which is suitable for any further work.

We are using this stage to prepare the environment (install Delve debugger, linters or package manager) but nothing more.

As you may note the magic has already started. There is no `COPY` or `ADD` of our source files into the image. Why we should put the source code into the image? During the development, it is very inconvenient. You have to always rebuild the entire image after each change in the source code. It is a slow process and generates a lot of garbage images (such images easily will take dozens of gigabytes during active development).

To solve this we mount the source code as a `volume` when running the container (take a look at the contents of the `docker-compose-dev.yml`).

Set up a target stage:

Mount the source code:

There is a small detail regarding a `command` option. As you can see we are using `go run` instead of `go build`:

During the development, it looks more logical because we run the app to make sure it works and see logs right away in the terminal (without extra `docker logs` steps).

Make sure a port for the debugger is exposed:

A couple of details to make debugger work:

`builder` + `exec` stages

The only goal of the `builder` stage is to build the target app. The `builder` uses the `env` stage as a base because we obviously need an appropriate environment to build our app.
We are `COPY` our sources only during the `builder` stage.

The single goal of the `exec` stage is to run the app.
Take a look at the entire docker-compose definition for production (`docker-compose-prod.yml`):

There is no noise at all. To run the production app you just need to use docker-compose up.

Makefile

We suggest using Makefile to define aliases for frequent and long commands.
In the example of a Makefile, you can find aliases to run the app locally, start a debugging session or run the app in production mode.

Debugging

A separate word to say about debugging. We believe a debugger is a must-have tool for the developer of any level. It simply saves you time. Therefore we put a lot of attention to make debugging of Golang apps frictionless for our developers.

As you may note we are using Delve with Goland. To make it work with Go Modules, you must have a `go.mod` within your app. Goland uses `go.mod` to map source files properly during the debugging session.
You have to define `working_dir` in the docker-compose as a parent folder for `go.mod` file (`/app` in our case).

If you are a fan of GOPATH (or you own a legacy project) you also can refer to our template with slight modifications:

• Remove `/app/go.mod`
• Follow comments in `docker-compose-dev.yml` file
• Make sure GOPATH is correct in the Goland project settings

And one extra quirk of Goland and GOPATH to make debugger work.
If your project is located, in the example, at `$GOPATH/src/go-docker-compose` at the host machine, you have to put the source code at `$GOPATH/src/go-docker-compose` inside the container. In other words, the part of the path relative to $GOPATH should match to allow Goland map sources during the debugging (while a value of $GOPATH can be different). See the `docker-compose-dev.yml` file for comments started with ‘To make it work with GOPATH’ to find out a working example.

Conclusion

We were happy to share our experience. Any feedback is appreciated.