Podman Machine Setup for x86_64 on Apple Silicon (run Docker amd64 containers on M1,M2,M3)

Photo by shawnanggg on Unsplash

Introduction

This post introduces a streamlined method to set up a Podman machine (QEMU) on Apple Silicon for running amd64 (x86_64) containers. We explore two approaches: multi-architecture support and fully emulated x86_64 machines.

Note: I will keep every script here and the explanation in this public repo too: https://github.com/guillem-riera/podman-machine-x86_64

Approach Overview

1. Mixed Mode, Multi-Architecture Support: This method enables support for multiple architectures, including x86_64, on a standard aarch64 machine. It maintains high performance for the native ARM images and has a performance impact on amd64 images. It operates on a base aarch64 machine and compatibility with amd64 images is as good as the package qemu-user-static can provide (I haven’t tested for full compatibility).
2. Full x86_64 Emulation: This offers maximum compatibility at the cost of slower performance. It’s a fully emulated x86_64 machine, which means that the containers are also run in fully x86_64 mode.

Recommendation: Always try the first approach (mixed mode) before considering the second.

Requirements

To get started, ensure you have the following installed:

• Homebrew
• Homebrew bundle
• Podman
• QEMU (automatically included as a dependency of Podman)
• jq

The required packages are listed in the Brewfile. Install them using:

brew bundle install

Setting Up

1. Multi-Arch Support on Current Podman Machine

This setup installs the necessary package qemu-user-static on your current machine.

This script facilitates this process:

export PODMAN_MACHINE_NAME=${PODMAN_MACHINE_NAME:-podman-machine-default}

### Stop all podman machine instances
ALL_PODMAN_MACHINES=$(podman machine list | awk '{ print $1 }' | tr -d '*' | sed 1d | tr '\n' ' ')
for PODMAN_MACHINE in ${ALL_PODMAN_MACHINES}; do
    podman machine stop ${PODMAN_MACHINE}
done

### Start the target podman machine
podman machine start ${PODMAN_MACHINE_NAME}

### wait for the podman machine to be running
PODMAN_MACHINE_STATUS=$(podman machine inspect ${PODMAN_MACHINE_NAME} | jq -r '.[].State')
while [[ "${PODMAN_MACHINE_STATUS}" != "running" ]]; do
    echo "[Info] Waiting for podman machine '${PODMAN_MACHINE_NAME}' to be running, current status: ${PODMAN_MACHINE_STATUS}..."
    sleep 1
    PODMAN_MACHINE_STATUS=$(podman machine inspect ${PODMAN_MACHINE_NAME} | jq -r '.[].State')
done

### Now that the podman machine is running we can install the package
podman machine ssh "${PODMAN_MACHINE_NAME}" 'sudo rpm-ostree install qemu-user-static'

### Stop the podman machine to apply the changes
podman machine stop ${PODMAN_MACHINE_NAME}

### Start the podman machine again
podman machine start ${PODMAN_MACHINE_NAME}

echo "[Info] Done. You can now run multi-architecture images in ${PODMAN_MACHINE_NAME}."

Podman can now run multi-architecture images with performance impacts limited to x86_64 containers.

How it works?

This bash script automates the setup of multi-architecture support for an existing Podman machine. Here’s a summary of how it works:

1. Setup: It sets the PODMAN_MACHINE_NAME variable, defaulting to "podman-machine-default" if not already specified.
2. Stopping All Podman Machine Instances: The script lists all existing Podman machines, excluding the header line and any active (marked with an asterisk) machines. It then stops each of these machines to ensure a clean setup environment.
3. Starting the Target Podman Machine: It starts the target Podman machine specified in PODMAN_MACHINE_NAME.
4. Waiting for the Machine to Run: The script continuously checks if the target Podman machine has reached the “running” state. It waits in a loop, checking the machine’s status every second.
5. Installing the Package: Once the target machine is running, the script remotely connects to it via SSH and installs the qemu-user-static package using sudo rpm-ostree install. This package is crucial for enabling multi-architecture support.
6. Restarting the Podman Machine: After the installation, the script stops the Podman machine to apply the changes and then starts it again

2. Full x86_64 Emulation Setup

Note: Follow this step only if the first solution doesn’t meet your needs.

Creating a new emulated Podman Machine (x86_64)

The following script creates a podman machine and alters it to make it an x86_64 machine (using QEMU):

# Setup the podman machine for x86_64 (QEMU), supports only Apple Silicon (Mx) Macs

# Keep all shell arguments in a variable to pass to the podman machine init command:
EXTRA_ARGS=${EXTRA_ARGS:-$@}

## 1. Download Fedora CoreOS image for x86_64 (QEMU)
PODMAN_X86_64_MACHINE_NAME=${PODMAN_X86_64_MACHINE_NAME:-x86_64}
PODMAN_X86_64_MACHINE_NAME_EXISTS=$(podman machine list | grep ${PODMAN_X86_64_MACHINE_NAME} | wc -l | tr -d '[:space:]')
PODMAN_QEMU_IMAGE="fedora-coreos-39.20231101.3.0-qemu.x86_64.qcow2.xz"
DOWNLOAD_DIR=${DOWNLOAD_DIR:-.}

if [ ${PODMAN_X86_64_MACHINE_NAME_EXISTS} -lt 1 ]; then
    curl -C- -O "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20231101.3.0/x86_64/${PODMAN_QEMU_IMAGE}"
    podman machine init --image-path ${DOWNLOAD_DIR}/${PODMAN_QEMU_IMAGE} ${PODMAN_X86_64_MACHINE_NAME} ${EXTRA_ARGS}
else
    echo "[Info] Machine ${PODMAN_X86_64_MACHINE_NAME} already exists. If you want to recreate it, run 'podman machine rm ${PODMAN_X86_64_MACHINE_NAME}'"
fi

## 2. Change machine settings

### Get the machine config file name
machineConfigFile="$(podman machine inspect ${PODMAN_X86_64_MACHINE_NAME} | jq -r '.[].ConfigPath.Path')"

### Change the QEMU binary to x86_64
sed -i '' 's/qemu-system-aarch64/qemu-system-x86_64/g' ${machineConfigFile}
### Change the firmware to x86_64
sed -i '' 's/edk2-aarch64-code/edk2-x86_64-code/g' ${machineConfigFile}
### Delete the additional UEFI firmware file (ovmf) and the preceding '-drive' option. The '-drive' option is in a line above the line containing the path to 'x86_64_ovmf_vars.fd'. Both lines must be deleted, but other -drive options must be kept.
#### using sed to match 2 lines: '-drive' followed by 'x86_64_ovmf_vars.fd'
sed -i '' '/-drive/{N;/x86_64_ovmf_vars.fd/d;}' ${machineConfigFile}
### Delete the HVF (Hypervisor Framework) acceleration, which is only available for macOS. This are also 2 lines: '-accel' followed by 'hvf'
sed -i '' '/-accel/{N;/hvf/d;}' ${machineConfigFile}
### Delete the TCG acceleration, which seems to work only for Alpha and ARM architectures. This are also 2 lines: '-accel' followed by 'tcg'
sed -i '' '/-accel/{N;/tcg/d;}' ${machineConfigFile}
### Change the machine type to q35
sed -i '' 's/virt,highmem=on/q35/g' ${machineConfigFile}
### Change the cpu type from 'host' to 'qemu64'
sed -i '' 's/host/qemu64/g' ${machineConfigFile}

How it works?

This script is designed to set up a Podman machine specifically for x86_64 architecture on Apple Silicon (Mx) Macs by modifying the QEMU template that podman generates when it creates a new machine.

Here’s a summary of its functionality and workflow:

Shell Arguments: The script stores any arguments passed to it in the EXTRA_ARGS variable, which will later be used in the podman machine init command.

Downloading Fedora CoreOS Image for x86_64 (QEMU):

• It sets a default name for the Podman x86_64 machine (PODMAN_X86_64_MACHINE_NAME) and checks if a machine with this name already exists.
• If the machine does not exist, the script downloads the specified Fedora CoreOS image for x86_64 using curl.
• After downloading, it initializes a new Podman machine with this image and any extra arguments provided.

Changing Machine Settings:

• The script retrieves the configuration file path of the newly created Podman machine.
• Several modifications are made to the machine’s configuration file to adapt it for x86_64 emulation:
• Changing QEMU Binary: Updates the QEMU binary from qemu-system-aarch64 to qemu-system-x86_64.
• Changing Firmware: Adjusts the firmware from edk2-aarch64-code to edk2-x86_64-code.
• Removing UEFI Firmware File: Deletes lines related to the UEFI firmware file (x86_64_ovmf_vars.fd) and its preceding '-drive' option.
• Removing HVF Acceleration: Eliminates the Hypervisor Framework (HVF) acceleration settings, as they are only available for macOS.
• Removing TCG Acceleration: Removes TCG acceleration settings, which are typically for Alpha and ARM architectures.
• Changing Machine Type: Updates the machine type from virt,highmem=on to q35.
• Changing CPU Type: Changes the CPU type from host to qemu64.

Conclusion

The podman offers a convenient way to run x86_64 containers on Apple Silicon, but you have to do extra steps to enable that.

Whether you require high performance or maximum compatibility, these methods provide a flexible solution to meet your containerization needs.

This is possible because QEMU, the underlaying virtualisation and emulation tool is really awesome!.

Alternatives

If you are looking for a very high performance and fully open source alternative to Docker Desktop that supports x86_64 / amd64 architecture with Rosetta, check my newer post on colima:

The most performant Docker setup on macOS (Apple Silicon M1, M2, M3) for x64 / amd64 compatibility.