Building x86_64 Docker Containers on Apple Silicon

Chris Torris Olsen
Dec 4, 2020 · 10 min read

I recently bought a new M1 Macbook Pro. My old one, a mid-2012 Retina, recently took its final breath, so there was no time to wait. For developers, the M1 is definitely not ready, but I decided to live with a few months of difficult transition rather than buy something that would soon be outdated.

One of the major components that at the time of writing don’t work is Docker, although it’s apparently close. For many of us, having Docker work is critical to our workflow, and I wanted to see if I could make that happen before Docker is officially ready. Also, just because Docker is released for the M1 doesn’t mean we can use containers meant to run on x86_64 seamlessly. The last part of this article is likely to be relevant for a good while, and understanding how emulation will support my work in the future is useful.

The promised land.

I’ll also note that my approach is unapologetically purist. We didn’t buy a laptop with a fancy new architecture to have it polluted by binaries from a world we left behind. Aside from what’s running inside your containers, and the Docker client, everything is running on the M1 without Rosetta 2.

Part 1: The Docker client

Install Docker for Mac. It’s going to fail if you start it, but that’s fine. You now have the command line tools necessary, and they work. That’s it, you’re done.

Part 2: Running a virtual machine

Docker runs fine on other ARM-based systems, such as Ubuntu Focal. That’s what I’ll use here.

It’s about to get a little gnarly.

There are several small projects that wrap the new virtualisation framework in Big Sur. I landed on vftool. Why? Because it turned out to be easy to avoid having a full XCode install to compile and run it, which is an overhead I’d like to avoid.

First, we want a folder to keep all our stuff in. Make sure you have the XCode command line tools available, and then download and compile vftool .

Almost there, but not quite. We have vftool installed, but if you run it with a kernel at this point, you’ll get an error:

To give it the entitlement, we need to sign the compiled binary. For that, you need a self-signed certificate. Open Keychain Access and use the Certificate Assistant to create one:

Clicking that will open a dialog. Set the certificate type to Code Signing, copy whatever is in the Name field, and create it.

Congratulations, you now have a working virtualisation wrapper.

You can run whatever Linux distribution you want, but I chose to run Ubuntu Focal’s cloud images (thanks, droidix). We need a kernel, the initrd, and the disk image itself. We also need to decompress the kernel and the disk image.

We also want to resize the disk image, otherwise we’ll run out of space. This would be easy to do with qemu-img , but it doesn’t compile natively on M1 with Homebrew. As I wrote earlier, this is a purist approach, so we’ll just use dd . It’s hacky, but it works. Let’s give it, say, roughly 20 gigs. (We’ll resize the actual partition later.)

Perfect. Now, let’s run this thing, but without specifying the root file system for now, so we can make some changes.

Run that, and note the TTY it’s connected to, which in this case for me was /dev/ttys009 . Open a second terminal window and connect to it.

Copy and paste the following (again, with thanks to droidix). This will change the root password to root , and set up SSH and networking.

Optionally, while you’re at it, add your SSH key for ease of access later.

We’re done here, so let’s make that known to the VM.

Go back to the first terminal window and hit Ctrl+C to kill the VM.

This time, run the VM with a root. I’ve also kicked up its allocated memory to two gigs for the moment, which seems to work fine for the purposes I’ve put it through so far. Change the -m flag to whatever suits you.

Again, note the device, and use screen to connect to it like above. You should be met with an Ubuntu login prompt, for which both the username and password is root . If you set up your SSH key in the previous step, you can type hostname -I to get the IP address of your virtual machine, and ssh into it with the root user. This allows you to detach screen (hit Ctrl+A then D) if you find that annoying to deal with — the VM will happily live on without it attached. Either way, we now have Ubuntu for arm running in a virtual machine on M1.

Remember I mentioned we need to resize the filesystem? Let’s get that done now.

That’s it!

Part 3: The Docker daemon

Now, from inside the VM, we want to set up the Docker daemon. Docker has pretty good installation instructions, so you should read those. If you don’t want to do that, just do the following:

Note: At this point, my VM and my VPN (Mullvad) were not happy about each other’s presence, and the VM could not reach the internet unless I disconnected the VPN. Let me know if you find a solution.

At this point, Docker is up and running. The installation suggests you run the hello-world image, which you can do now if you want to.

Now, let’s set up the Docker daemon to accept connections from the outside world. The instructions are here, but again, you can follow my instructions if you’re lazy.

Let’s also make sure it’s listening.

The Docker daemon is set up and running. Make sure you get the IP address of the VM with hostname -I, and let’s move on to the client.

Part 4: Building x86_64 Docker containers

Now to the promised land. Make sure you have a terminal window in the VM (denoted with #) and in macOS (denoted with $ ) handy. First, make sure your Docker client knows where to find your server, replacing the IP address below with whatever you got from the VM.

You can verify the connection by checking that the hello-world image used above pops up.

You can stop here if you just want to build images for ARM. Congratulations, you’ve done it. If Docker for Mac is updated for M1 in two days, you’ll probably feel like you just wasted a lot of time. If you want more, let’s venture on. It should be useful even after Docker is updated.

Buildx is an experimental feature in Docker. To enable experimental features in the client, we need to set the experimental flag to enabled in Docker’s configuration. If you haven’t made any changes to your Docker configuration, just run the following line. If you have, do it manually, or at least absolve me of any and all responsibility for mucking it up.

Let’s create a very minimal container that we can use for testing.

Let’s build and run that for arm to see it running. You can open it in the browser on port 8000 with the IP address you got above.

Note: The container did not quit gracefully sometimes. If that happens to you, go to the VM terminal window you have open, and type docker kill $(docker ps -q) , which kills all running containers.

If you try to build it for x86_64, things don’t go as well:

What we need is to emulate x86_64 instructions so we can get a container built. Artur Klaser has an article on getting this up and running for computers building for arm, but we want the opposite. We need to install a couple of things to make this work. Let’s turn back to the VM to get that running. This could take a while, so grab a drink while you’re at it.

We also need to recreate the builder so buildx understands that it has new platforms to target, and then use that one as our default:

That looks pretty good! Let’s try it out. You’ll probably notice that this is a bit slower than the first run, due to emulation. The --load option below makes sure the image doesn’t just get discarded.

Let’s run the images we built:

And we’re done! This only works because the base image is available for multiple architectures, but many popular Docker images are. You’ve now got two images that built for x86_64 and arm respectively, and both run in your virtual machine — albeit one with emulation. If you run the newest image normally and visit the app in the browser, it works just fine.

End notes

I’m somewhat in awe of the amount of technology that lies beneath making this happen. We’ve got a brand new chip in the M1, containerisation, emulation, virtualisation — the amount of developer hours that have gone into making something like this work is astounding.

There are still going to be some quirks in this setup. qemu does not work perfectly, and most of its use in Docker seems to be from people on x86_64 architectures building for arm, so any bugs the other way are likely to just be popping up now that M1s are being distributed. But if you’re building images that use interpreted code, with few or no compiled dependencies, this should work just fine! Adrian Mouat has a blog post on the Docker Blog that explains further, and has some helpful alternatives to explore, including cross-compilation.

PS: A much easier solution to this is to run an x86-based virtual machine on the free tier of a cloud service of your choice. To do that, install Docker, configure your daemon to receive connections from the outside, and set DOCKER_HOST in your environment appropriately. No other options are required, as the machine you’re building on doesn’t know any better.

The Startup

Get smarter at building your thing. Join The Startup’s +799K followers.

Chris Torris Olsen

Written by

Tech lead and International Relations student. https://twitter.com/ctolsen

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +799K followers.

Chris Torris Olsen

Written by

Tech lead and International Relations student. https://twitter.com/ctolsen

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +799K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store