Installing 64-bit Fedora on the Raspberry Pi 4

Hacker; noun. One who makes furniture with an ax.

This is a hack.

This is a hack because as of the writing of this piece, we have two excellent halves of a solution, and I just need something that works.

On one hand, the Raspberry Pi Foundation have

• A pretty remarkable 1.5GHz quad-core, 4GB memory, 64 bit computer with enough stuff on the motherboard to qualify for a desktop, in a $50 package, called the Raspberry Pi 4b.
• A Linux kernel that supports Pi 4b features.
• A Raspbian (Debian derivate OS) image, which contains a boot loader that can controls the many hardware features of the Raspberry, just like a PC BIOS does. It also contains ready 32-bit and 64-bit Linux kernels, that will happily boot on any Linux distribution.
• Sadly, they don’t maintain a 64-bit “userspace”, because they’re small and don’t want to support two userspaces, one for their 32-bit stuff and one for their 64-bit capable stuff.

On the other hand, we have the Fedora project. An enterprise-grade operating system user-space built natively for 64 bit arm, with a massive catalogue of enterprise software, some Seriously Cool Stuff that can be built off it (I’m looking you right in the eye, Kubernetes+etcd straight from the OS repos). Sadly, Fedora does not currently support the Pi 4b, its u-boot boot loader doesn’t fit the Pi 4b like a glove, and the kernels as of this moment do not run stable on the Pi 4b model or support some of its hardware.

This guide will show how to put the above together in minimally working order. Because I need to repeat this a lot, I cut some Ansible code to do it all.

Some prep work first:
We will need one or more running PCs to fulfil the following roles (I assume you are running Fedora or a derived OS; it will work on Debian/Ubuntu just as easily, but you’ll need your nerd license to adapt it a bit):

1. base: A place to run some commands (an Ansible playbook).
2. builder1: A place to assemble some images. All up, prep at least 16GB of space or so. This will be done as a user that can sudo to root.
3. flasher1: A place you can insert an SD card. This too will need a user that can sudo to root.

The same machine can be all three (but you can also use AWX/Tower for base, a virtual machine for builder1 and a Fedora laptop for flasher1, if you’re thus inclined. It doesn’t really matter.)

1. Install ansible and git:

[mikishapiro@base ~]$ sudo dnf -y install ansible git

2. Grab the code:

[mikishapiro@base ~]$ git clone https://github.com/mikishapiro/armbuild.git

3. Modify your default Ansible inventory file (armbuild/inventory), with the settings you want.

This includes naming which machine(s) will fill the builder1 and flasher1 roles, some configuration for the resulting Raspberry Pi Fedora image and so forth.

You will also need to plug your SD card in the flasher1 machine, run

[mikishapiro@flasher1 ~]$ sudo lsblk

… and figure out from the output what device it is. It’ll probably be a /dev/mmcblk* device if it’s sitting in an onboard SD card reader, or a /dev/sd* device if it’s sitting in a USB adapter. I use a USB adapter, and it appears as /dev/sdb on my system.

4. We’ll need to set up a bit of ssh access to allow Ansible to do the heavy lifting for us.

Ensure your user is in the ‘wheel’ group and can sudo to root on builder1/flasher1. To simplify this writeup, I will assume you can sudo to root without needing to provide a password (uncomment the %wheel line containing NOPASSWD in /etc/sudoers and comment the original one to achieve this).

5. Ensure your user on base and builder1 has an ssh key generated. Run ‘ssh-keygen’ command and create one with a blank password if not.

6. Ensure your user on base can ssh to builder and to flasher by logging on to base and typing

[mikishapiro@base ~]$ ssh-copy-id builder1
[mikishapiro@base ~]$ ssh-copy-id flasher1

Also, you will need to ensure your user on builder1 can ssh to root@flasher1.

builder$ ssh-copy-id root@flasher1

That should be it.

Note: If you use any of this in any kind of production environment, you will need to make this method more secure, by storing sensitive configuration items (like password hashes or a public ssh key) in Ansible Vault (or AWX/Tower), or by avoiding the ssh to root operations I use. This is outside the scope of this writeup.

That’s the prep. Now just run the deploy.yml playbook:

[mikishapiro@base ~]$ cd armbuild
[mikishapiro@base armbuild]$ ansible-playbook -i inventory deploy.yml

You can add -vvv to the second command if you like gory debug.

If everything is in order, that should be it. Some 10–15 minutes after the downloads are complete, you should have a flashed SD card (assuming USB2 transfer speeds). Transfer your SD card to your Raspie and power it up.

Re-running the playbook (with same or different SD card) will not re-perform the lengthy download operations if not necessary, and if you use the lazy=true configuration item in the inventory, it’ll recycle the existing unarchived images as well, saving more time.

Go ahead and have a look at the Ansible code to see what it does — the core logic is in the deploy.yml playbook, while some of the low level operations are inside the arm-build role, in the tasks/*.yml files related to the instructions it knows how to handle.

I’ve tested this using a pre-release build of Fedora Server raw-xz (aarch64) version 32 from here: https://www.happyassassin.net/nightlies.html and Raspbian Lite from here: https://www.raspberrypi.org/downloads/raspbian

In a nutshell, the playbook (running on base) will

1. Download and unarchive both OS images (to builder1), but without deleting the archives. There’s a bit of acrobatics in there for the Raspbian image to resolve its redirect URL, find out the real filename and save the image with the correct filename. This saves re-downloads.
2. On builder1, mount via loopback and modify the unarchived images inside the unarchived files themselves. In the case of aarch64 Fedora there is LVM on the image (this is different from 32-bit arm Fedora 32, which does not), and we need to install and use kpartx on builder1 to allow lvm to see the contents so you can mount the filesystem on the logical volume inside. This will happen automatically.
   Note: If the builder1 system’s own LVM volume group is also named fedora (some, but not all, fedora versions do this), this playbook won’t be able to reference the contents of the loop device just by mentioning volume group name. You’ll need to do that bit manually, rename some volume groups, or fix some playbooks.
3. On the Raspbian image, in the boot partition (which contains all of bootloader, Pi 4b firmware configuration and kernels) we will tweak the firmware config (config.txt) to use 64 bit mode (this makes it boot using the kernel named kernel8.img in the same directory), and the kernel command line (cmdline.txt) to remove the once-off Raspbian repartitioning init directive (making the kernel default to /sbin/init).
4. On the Fedora image, we’ll change PermitRootPassword to yes in /etc/ssh/sshd_config, inject an ssh key to /root/.ssh/authorized_keys and/or a root password hash into /etc/shadow if specified in the inventory. Since we’ll be using the Raspbian kernel, we also need to bring from the Raspbian root partition the drivers that were compiled with it and for it. We will copy over /lib/modules/* from the Raspbian root partition to the fedora root partition, and bring along some modprobe configuration as well to prevent the wrong drivers from being loaded. We will also configure a network interface on the Fedora root (which appears in aarch64 Fedora as eth0, hence /etc/sysconfig/network-scripts/ifcfg-eth0) with IP settings you can set in the inventory. It can also disable some services to make the whole thing a bit quicker to come up.
5. Once the large image files on builder1 contain what needs to go on the SD card, base will log on to flasher1 and do a bit of advance housekeeping, such as wipe the first 300MB of the SD card using /dev/zero to avoid any previous contents on the SD card from tripping up the use of the parted tool.
6. base will now log on to builder1 and using a ‘dd | ssh root@flasher1 dd’ command, stream the part of the Raspbian partition making up the Master Boot Record (which contains the Raspbian image partition table) as well as the boot partition to the SD card.
7. base will now log on to flasher1, and remove the second (root) partition from the partition table (the data of that second partition isn’t actually on the SD card, we only streamed the MBR and the data for the first partition). It will then create a new partition on all the remaining space on the SD card, which, for the default Raspbian kernel to function, has to be ext4 and cannot be xfs. A new filesystem is then created on it, and it is mounted. base then proceeds to stream all the files from the Fedora image partition, still mounted via loopback on builder1, to flasher1 using a “tar cfp — . | ssh root@flasher1 ‘cd <sd>; tar xvfp -’ ”command.
8. Some last rites — a chmod 755 on the root directory of the SD root partition on flasher1, tidy umount of the file images on builder1, kpartx & losetup loopback device cleanup on builder1 — and we’re all done. You’re set to re-run it.

I hope you found this helpful.

In my next write-ups, I will be covering

• How to natively compile a kernel directly on the aarch64 Fedora v32 system produced above (instead of cross-compiling or using an emulator), using the kernel source tree provided by the Raspberry Pi foundation (which contains full support for the Pi 4b features), and with the fancy stuff we will need for OKD and COSA.
• How to build the Fedora CoreOS Assembler (COSA) on this system, so we can assemble an aarch64 Fedora CoreOS image.
• How to assemble an actual aarch64 Fedora CoreOS image.
• How to build OpenShift Kubernetes Distribution (OKD) version 4.4 kube go components on our aarch64 system.

Have fun, and don’t forget to be awesome :)