Get your updates right, but don’t leave old files behind

By Thibault Jamet (Backend Developer)

After 10 years of existence, we have in leboncoin a considerable code base. As with any other project, our developers are adding, renaming or removing components.

During our development process, several developers are going back and forth between feature branches prior to be merged.
When leaving a feature branch, we need to remove any built object added by this feature.

As we are using git for version control, the simplest way would be to use the git clean -dxf command, but this would imply building everything again which is a waste of time (on our legacy monolith, a fresh build requires around 20 minutes).

We will see how to achieve the same goal, with Make, without needing to rebuild anything that has changed.

WHAT IS MAKE?

GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program’s source files.

Make gets its knowledge of how to build your program from a file called the makefile, which lists each of the non-source files and how to compute it from other files. When you write a program, you should write a makefile for it, so that it is possible to use Make to build and install the program. ( Source)

With awareness of both how to generate a file (the target) and what that file needs (its dependencies), make can help save valuable time and prevent you from having to rebuild a target whose dependencies did not change since the last build.

Although make is great at saving time, it takes some expertise to manage efficient work during code cleanup phases.

THE MAKE CLEAN TARGET

Most projects come with a make clean target providing a developer with the ability to remove any non-source file the make program generated. Even the reference itself instructs how to clean build output, based on the list of buildable targets:

The whole Makefile might be close to:

It is then possible to run:

It also works when adding targets:

However, when targets are removed (e.g. when you are moving outside of a feature branch adding a program), make won’t be able to clean your working directory properly:

IMPROVING THE MAKE CLEAN TARGET

As we noticed, our current implementation of the build is not reproducible, hence some files in the bin are still available but not rebuildable after a git clone.

To solve this issue, the first approach would be to clean the whole repository after each checkout with git clean -dxf . However this can be very inefficient if your build time is high.

Another option is to add a simple script in charge of cleaning previously buildable targets (./cleanup.sh):

Integrate it in the Makefile:

Test the change:

CONCLUSION

Whenever you’re considering performing an update either with make as described here, or with Puppet, Ansible or other configuration management tools, don’t forget to remove anything redundant to prevent the famous it works for me pattern.

You can find the illustrations of this article hosted on GitHub: https://github.com/leboncoin/blog-clean-examples