Creating reproducible release tarballs

This weekend I was going to create a new release for tilda. I have documented the release process in the tilda repository and usually building a new release is relatively straight forward. I create a release tarball and use that to build the Debian source and binary packages. Then I verify that there are no errors and no lintian warnings. As the last step I upload the package to Debian mentors to get it uploaded into the Debian unstable repository by a sponsor.

This weekend the process turned out to be a little different. I wanted to create a patch level release from tilda 1.5.0 to tilda 1.5.2. After I had created the new Debian package, I was comparing it with the 1.5.0 version of the package to determine what changed and to ensure that it only contained a minimal amount of changes acceptable for a patch level release.

To compare the newly build package with the existing package in the repository you first need to download the source package using apt-get source and then you can use debdiff to compare it with the newly build package:

apt-get source tilda
debdiff --diffstat \
  tilda-1.5.0-1.1.dsc tilda-1.5.2-1.dsc > tilda.1.5.2.debdiff

The results were a bit surprising. The debdiff contained multiple changes to the build system and the generated glade-resources.c source file had completely changed:

diffstat for tilda-1.5.0 tilda-1.5.2
[...]
 Makefile.in | 520 ++++++ — — 
 aclocal.m4 | 193 + — -
 build-aux/compile | 13 
 build-aux/depcomp | 8 
 build-aux/install-sh | 36 
 build-aux/missing | 16 
 configure | 211 ++ — 
 [...]
 src/glade-resources.c | 2369 +++++++++++++++++-----------------[...]

The reason for these changes were, that I had created the release tarball for tilda 1.5.0 on an Ubuntu 18.04 system and since then I have upgraded to Ubuntu 20.04 and now I used that to create the tilda 1.5.2 release tarball. Ubuntu 20.04 contains automake 1.16 instead of automake 1.15 which explains the changes to the build system and it contains a newer version of glib-compile-resources which changed its behavior and now outputs the compiled data as a string.

So it turns out that the system on which a release tarball is created affects the contents of the release tarball and thus also the structure of the final package that is built from it. So how can we create a reproducible release tarball that is always build from the same environment?

It turns out this is a very good use case for docker. We can create a docker container from an Ubuntu 18.04 image and then we will always have the same environment no matter if the host system is an Ubuntu 18.04 or 20.04.

We first create a new container based on Ubuntu 18.04:

docker run --name ubuntu-18-04 -it ubuntu:18.04 bash

Then inside the container we install everything needed to build tilda and finally we clone and build tilda:

#! /usr/bin/env bash
# build-tarball.sh
apt update
apt install -y git libpcre2-dev
# Enable all deb-src entries
sed -i -- 's/#\s\?deb-src/deb-src/g' /etc/apt/sources.list
apt update
apt build-dep -y tilda
git clone https://github.com/lanoxx/tilda
cd tilda
git checkout tilda-1.5.2
mkdir build
cd build
../autogen.sh --prefix=/usr
make distcheck

Normally the extra libpcre2-dev package should be installed by build-dep but the dependency is not correctly declared by the tilda-1.5.0 package and so its not being pulled in.

After these steps there should be a tilda-1.5.2.tar.gz tarball under /tilda/build/tilda-1.5.2.tar.gz.

The final step is to copy this from the container to your host system:

docker cp ubuntu-18-04:/tilda/build/tilda-1.5.2.tar.gz .

Now we can use the release tarball to build our Debian package.

The above steps can be automated even further. If we save the above steps in a script (e.g. build-tarball.sh) and mount that into the container:

docker run --name ubuntu-18-04 \
    -v `pwd`/build-tarball.sh:/build-tarball.sh \
    -it ubuntu:18.04 bash -c "./build-tarball.sh"

After that we can copy the tarball to the host using the docker cp command from above.

Update: In the previous version I also installed tilda in addition to git and libpcre2-dev, in the build-tarball.sh script, but it turned out that the tilda package was not needed to build the release tarball, so I have removed the tilda package from the list of installed packages.