Building ARM Docker images with GitLab-Runner
There are multiple ways of building ARM docker images and there is not always a need to have an ARM computer or server in place to do so. Using QEMU ARM architecture can also be emulated on a normal AMD64 computer or laptop. Only when having to compile a lot of packages on ARM, the best advice is to not use emulation since it will take forever. Also, it is always important to keep in mind which ARM platform to target, be it armv6l, armv7l or aarch64.
Why build ARM Docker images?
With the rise of the Raspberry Pi and its evolution from a weak, constrained resource computation board to a pretty powerful single-board computer with very wide adoption, ARM Docker images can more and more be used to power various applications. This can be almost everywhere since a Raspberry Pi is available for little money and can even be used in a mobile project powered by nothing more but a USB power bank. Using Docker, one can build isolated images containing everything necessary to run everywhere with very little configuration. Installing software is almost as easy as installing an app on our smartphones.
The ARM architectures (Yes, plural!)
When building software and Docker images for ARM it is important to know which specific architecture version to target. Other than the well-known x86 (32-bit) and AMD64 (obviously 64-bit) which mainly differ in terms of the memory address lengths (32-bit vs 64-bit), ARM does have various different versions and, in some versions, also the two different address lengths. These different versions combined with a naming scheme that isn’t quite obvious makes it a little more complicated to tell the different versions apart. Let me try to shed light on this by explaining the different versions in a little more detailed way. Since Raspberry Pis are already widely adopted, I also try to mention which ARM architecture is used in which Raspberry Pi model.
Even though there are older versions than v6, I decided not to go into further detail for them since, in my opinion, they are not really relevant for this blog post.
ARMv6 CPUs can be found in RPi 1 and RPi Zero. It’s a 32-bit architecture and due to its age, most CPUs found on single-board computers do not really provide high performance. Therefore, I do not really suggest trying to run Docker on such a platform since the experience will be most disappointing. Nevertheless, there is an official arm32v6 profile on Docker hub giving access to often used images directly build to support this platform.
ARMv7 CPUs is the architecture that can be found in RPi 2 (before v1.2). It’s a 32-bit architecture that delivers in most cases better performance than ARMv6 and is experiencing a wider adoption across different software packages and Docker images, which can be found in the official arm32v7 profile on Docker hub.
Here I find it necessary to mention that also newer RPis such as RPi 3 or RPi 4 do show armv7l as the machine hardware name when they are running a 32-bit Linux operating system.
ARMv8 CPUs can be found in multiple single-board computers such as the RPi 2 (1.2), RPi 3, RPi 4, RPi Compute Module 3+4, and the RPi 400. These CPUs are capable of running 32-bit as well as 64-bit operating systems. When running a 32-bit OS we often talk of aarch32, which is compatible with ARMv7 (this is also shown as armv7l machine hardware name). When running 64-bit we can see the aarch64 machine hardware name. The Docker images for this specific version can be found in the official arm64v8 profile on Docker hub.
ARM version summary
In order to figure out what kind of architecture your Docker images need to support, you need to know not only the ARM version of your CPU but also what version of OS you are running. To figure out your machine hardware name simply call:
Or use neofetch which shows the version in the OS line.
The following table does show the different versions and “uname -m” outputs:
Depending on the machine hardware name you can use different Docker images:
armv6l → https://hub.docker.com/u/arm32v6/
armv7l → https://hub.docker.com/u/arm32v7
aarch64 → https://hub.docker.com/u/arm64v8/
The challenges when building ARM Docker images
Even though ARM is almost everywhere, be it in our smartphones or in various single-board computers, the platform didn’t yet see wide adoption in desktop computers or laptops. Not that there aren’t any ARM-powered devices in that field, it’s just that most of us still work on Intel or AMD chips. Even though Apple already launched its apple silicon M1 chips, the rise of ARM in the field of desktop and laptops is still to come. Therefore, one of the biggest constraints when building ARM Docker images is the different CPU architecture of our computers. In this blog post, I will explain how cross-platform builds can be done on Intel or AMD-based machines and how to install GitLab-Runner on a Raspberry Pi.
Due to emulation, we can still run and build ARM Docker images using our daily working machines powered by Intel or AMD CPUs. One important thing to mention right away is that if you are planning to compile a lot of software, you will not be satisfied by the performance of ARM emulation on an AMD64 platform. In certain circumstances, even a Raspberry Pi 4 with its quad-core Cortex-A72 Processor will perform better than an AMD Ryzen 3900X with 12 cores. This is mostly the case when compiling a lot of packages. Nevertheless, in some situations emulation is perfectly fine and suitable, and sometimes we might just not have a spare Raspberry Pi board for a dedicated ARM build server.
Setup ARM emulation
Setting up ARM emulation on an AMD64 Linux system is quite straightforward and doesn’t even require installing any packages. What we need for this step and also later on to build ARM docker images is a running Docker instance. Anything further can be achieved by something called miscellaneous Binary Formats. Using binfmt_misc we can register our emulators for different CPU architectures right into the Linux kernel. The guys from Hypriot prepared pretty handy docker images that help us to do so.
Update March 1st 2022:
Since Hypriot does not further maintain its image, I propose to use the qemu-user-static image by multiarch which is still maintained.
The only thing we need to do to register the emulator is to run the following command:
The registered emulators are only available as long as the computer or server isn’t restarted. If we want this to “survive” a reboot, we can add the following command to our “/etc/crontab” file, calling the above command at every reboot of the computer:
At this point, we can already run ARM Docker images on our machine. To see whether it worked we can run the ARM64v8 Traefik image:
In a second terminal, we can then use the following commands to check the architecture:
Setting up GitLab-Runner to build ARM Images using GitLab CI/CD
Setting up a GitLab-Runner using Docker Compose is pretty straightforward. Creating a directory called “gitlabrunner” and within create a file called docker-compose.yml:
Put the following into your docker-compose.yml file:
Spin up the container using “docker-compose up -d”:
To configure the GitLab-Runner use the following command:
And then follow the instructions:
Find the registration token in the settings of your project (Settings → CI/CD → Runners):
After finishing the registration process, we go into our Docker Volume and edit the config.toml file:
We can then change the settings for the concurrent jobs to a number we like and the privileged mode to true:
That’s it. Due to the QEMU emulation, our newly installed GitLab-Runner is not only able to build AMD64 but also armv7l and aarch64 images. In order to do so, the only thing we need to do is to choose the right base Image from the Docker hub. In the next blog post, I will show how to do so and what is needed to set up a simple CI/CD pipeline on GitLab.
I hope I was able to explain the differences between the various ARM architecture versions and how to use the correct Docker base images to build for your ARM system. Using QEMU directly is only one of the various ways to build ARM Docker images and it gets more and more common to use docker buildx to do so. Especially when building cross-platform images for multiple architectures at the same time buildx can be a good fit. I might also write a blog post on how ARM images can be build using docker buildx. But for now, that’s it. If you liked this blog post, make sure to follow me and support my writing. Thanks for reading!
About the author
Remo Höppli is Co-Founder and Software Engineer at Earlybyte.
Earlybyte is an IT consultancy firm specialized in developing new digital solutions for companies around the world from digitalization to IoT solutions, close to the client and its business embracing agility.
Follow me on Twitter to get informed on new blog posts.
Remo Höppli - Co-Founder & Software Engineer - Earlybyte GmbH | LinkedIn
I'm a tech enthusiast working in my own Startup-Up company Earlybyte, which I founded with my study peers from ZHAW…
Docker Pirates ARMed with explosive stuff
We're proud to announce our 1.11.0 release of HypriotOS - the fastest way to get Docker up and running on any Raspberry…
Docker Image to register Qemu interpreters for ARMHF/AARCH64/PPC64LE/Risc-V in Docker Host w/ Linux Kernel 4.8+ …
The Docker Official Images are curated images hosted on Docker Hub. The main tenets are: See Docker's documentation for…