Three good practices for better CI/CD Makefiles

One interesting release engineering practice that has emerged recently is the use of Makefiles to implement continuous integration and delivery pipeline tasks. GnuMake as a technology has been around for a very long time, before its current re-emergence as some kind of universal CI/CD executor, it was mostly known for its use as a build tool for C and C++ software.

The main point of attractivity to Makefiles is that they provide a very consistent interface to define and execute the tasks needed to build, tests, release and deploy most software in a cloud-native context.

Like any piece of code, if written without much care for good coding principles, Makefiles can very rapidly grow in size, become very complex, and lose any sense of readability and maintainability. Going further in the post I will discuss three ad-hoc software craftsmanship practices that I have used to keep Makefiles simple, readable and usable: namespacing, composition and self-documentation.

Namespacing

Continuous delivery platforms divide pipelines into sequences of work units called stages (Jenkins, Gitlab) or steps (Drone), for example, a common delivery pipeline will be made of the following stages: build, tests, release and deploy.

To increase the readability of a Makefile, one could implement a naming convention that establishes an explicit binding between a target and a stage in a continuous delivery pipeline file where it is used to execute a defined task.

This creates a ubiquitous language ensuring that when reading a Makefile we will always know which stage of the pipeline calls a specific target. To implement this practice we simply use the following naming pattern “stage.task” to define every target in the Makefile.

Composition

There are many reasons why Makefiles become complex, unreadable and hard to maintain, one of the main reason is that targets handle too many tasks. The obvious solution to such a problem is to write Makefiles that respect software craftsmanship rules such as composition and single responsibility principle.

A target should have the responsibility of implementing the execution of one single task, but when multiple tasks need to be executed by calling one command in the CLI then a target could be implemented by the composition of many single-purpose ones.

For a project of significant sizes such as a monolith containing everything from a front-end UI to a backend API or a monorepo containing multiple services, a single Makefile approach might not be the right one. In that case, implementing composition using GnuMake file inclusion could solve many issues that could arise down the line from writing the entire build process into a single Makefile.

In a project where build tasks have been split into multiple makefiles, there is a simple way to include them all in the main makefile as illustrated below.

-include build/Makefile.*
  1. The leading “-” allows “make” to ignore errors if a file is missing.
  2. This line should be added at the top

Self-Documentation

The concept of the self-documenting code is a well-known software craftsmanship principle, it streams down from the notion that the code source itself is a document that tells both humans and machines how the system is supposed to behave.

When adapted to Makefiles, each target will be written in a way that a simple command will display the list of targets and the description of the task each of them executes. We could just type make, and get a list of available commands, together with their description into a terminal.

To achieve this, all targets should be preceded by a comment that starts with # and contains the description of the task it executes itself preceded by @.

For this, to function, we need to create a special target named “help as illustrated below and make it the target by default.

.DEFAULT_GOAL := help
#help:  @ List available tasks on this project
help: @grep -E '[a-zA-Z\.\-]+:.*?@ .*$$' $(MAKEFILE_LIST)| sort | tr -d '#' | awk 'BEGIN {FS = ":.*?@ "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

When the “make help” or the default “make” target is called, a list of available targets followed by a short description of the task they execute will be displayed on the terminal.

Further explorations

There are obviously many more good software craftsmanship practices to implement in Makefiles to achieve a high level of maintainability, readability, and usability, here are resources to consult in that purpose.

  1. Basics https://www.gnu.org/prep/standards/html_node/Makefile-Basics.html
  2. Best practices https://danyspin97.org/blog/makefiles-best-practices/
  3. Style-guide http://clarkgrubb.com/makefile-style-guide