RSoC: Porting Redox to ARM (AArch64) — 0x02

8 min readAug 5, 2018

*Getting closer to UEFI, HiKey960 and the kernel*

What am I going to do?

Two weeks ago, I blogged about @microcolonel’s plan to port Redox OS to the Armv8 Architecture and specifically to the 64-bit AArch64 execution state.

In this edition, I’ll blog about my personal explorations of the following ⌨️

Setting up the HiKey960 reference board (kindly offered by @microcolonel and ARM; thanks!) for Serial Communication to the host development system (Fedora Linux28)
Sanity checking @microcolonel’s current status of Redox OS. As a first step, and as explained in that doc, I compiled the whole of Redox for x86_64 on a Debian Stretch 9 host.
Building @microcolonel’s AArch64 Redox kernel development branch to get my first AArch64 entry into the Redox (micro) kernel!
Exploring the possibility of chain loading U-Boot from UEFI (i.e. Use U-Boot as an EFI payload) . For more details on this, look at this SuSE paper.
Getting on par with @microcolonel’s qemu based core kernel work on HiKey960 reference board. I will describe what works, where it fails, and the difficulties encountered.
Sharing some interesting community links, a Stanford course for Raspberry Pi (spoiler: it includes writing low level code in RUST 😍) and some insight on UEFI, U-Boot and OS Booting. A good read for anyone entering the playing field!

A summary of work done since the last blog posting

At the time of writing the previous blog the plan was to target the Raspberry Pi 3 (Cortex A53) as a development platform because of its availability, popularity and community. Sadly, it seems that Broadcom went through a lot of shortcuts while implementing this specific design, which means features like GIC are half-there or completely missing, like in this case.

After a discussion with @microcolonel, he proposed and kindly sent me a HiKey960 reference SoC from the awesome Linaro 96Boards initiative. The quality of this board is definitely a lot better than the Raspberry Pi and the documentation is detailed and open. Great stuff.

I am also excited as I wait for an Asus Tinkerboard, offered by another member of the Redox community: @stratact. Thank you so much for your contribution and I hope I can put it to use someday! Looking at you Redox kernel!!!

Board Preparation

The HiKey960 needs a UART to USB cable (such as the Mezzanine board from 96Boards or any other 1.8V-compatible cable; beware that is not your usual 3V3/5V Amazon TTL2USB!)

Pin 1/2 is GND. Then, we need to connect the Pins 11 (TxD), 13 (RxD) to their equivalent on the cable.

The colors aren’t universal but in my case they went like:

White — RxD from the USB side
Green — TxD (same as above)
Black — GND
Pink (I suppose trying to be red) — Vcc [leave that unconnected]
In case that you need there’s also an active low reset available on pin 6 of the 40 pin connector. A jumper from there to GND will reset the board and set it to fastboot mode (optional)

For reference, see above for a photo of this setup.

If everything goes well you will be welcomed with a beautiful boot sequence and kernel messages from the stock Android AOSP installation:

Sanity check the Redox build environment

Here is the summary of my process, mainly following @microcolonel’s How-To but improvising at a few different points:

Create a VM and install Debian Stretch 9
Install the required packages and libraries
Get the keys and install a Redox toolchain
Install Rust (nightly), cargo and xargo
Do a make fetch; make pull on the main repo
Get a Redox login prompt (x86_64) with make kvm=no vga=no qemu

Mission accomplished! 🎉

Compile the kernel for AArch64

What? Yes indeed! Thanks to @microcolonel’s this can be done (check the above mentioned How-To for detailed steps).Switch to origin/aarch64 branch

Summary:

Switch to the origin/aarch64 branch
Update the submodules (kernel and syscall)
Get the aarch64-elf GNU toolchain
Get @microcolonel’s modified rust compiler that is using llvm with added toolchain triplet support for the aarch64-unknown-redox
Compiling a patched version of U-Boot
Let rustup know about our new AArch64 Rust compiler
Build the kernel!

make ARCH=aarch64 CROSS_COMPILE=aarch64-elf- PLATFORM=qemu-system-aarch64-virt INSTALLER_FLAGS=”” build/kernel

Building UEFI for HiKey960 from source

Hikey960 currently only supports UEFI based booting using the Tianocore edk2 project. @microcolonel and I are exploring how we can get to the same boot flow by chain-loading u-boot from edk2 but that’s for later.

Code Preparation

Note: I only recently discovered that you can find pre-built binary UEFI images from Linaro but here are the steps I followed ot build from source for those who want to learn about it!

Create a working directory, let’s call it makeUEFIand get all the repositories needed in there:

It is important to get the current working branches in order to compile successfully. You may need the openssl-dev package as well, so keep that in mind.

git clone https://github.com/ARM-software/arm-trusted-firmware -b integration
git clone https://github.com/96boards-hikey/edk2 -b testing/hikey960_v2.5
git clone https://github.com/96boards-hikey/OpenPlatformPkg -b testing/hikey960_v1.3.4
git clone https://github.com/96boards-hikey/l-loader -b testing/hikey960_v1.2
git clone https://git.linaro.org/uefi/uefi-tools
git clone https://github.com/96boards-hikey/atf-fastboot

Let’s build!

Note: You may be able to find lot of the following instructions online, especially in the HiKey960 GitHub official documentation, as well a few Japanese blogs, where they are installing Debian on the HiKey. The problem is that in all of those sources, there’s always an error that stops you from following the steps successfully.

It feels almost like it’s an upside-down documentation (first tinkering with code and then trying to remember the steps involved!). By following the steps below you can be sure that everything is meticulously tested :)

Save the following bash script and execute it on your ${BUILD_PATH}

#!/bin/bash
# Install UEFI from source on HiKey960export BUILD_PATH=`pwd`cd ${BUILD_PATH}/edk2
ln -s ../OpenPlatformPkg # for the v1 of Hikey960, remove the following line from the tools/platform.config
# BUILDFLAGS=-DSERIAL_BASE=0xFDF05000BUILD_OPTION=DEBUGexport AARCH64_TOOLCHAIN=GCC5
export UEFI_TOOLS_DIR=${BUILD_PATH}/uefi-tools
export EDK2_DIR=${BUILD_PATH}/edk2EDK2_OUTPUT_DIR=${EDK2_DIR}/Build/HiKey960/${BUILD_OPTION}_${AARCH64_TOOLCHAIN}cd ${EDK2_DIR}# Build UEFI & ARM Trust Firmware
# Again, please use gcc-5.3 to build${UEFI_TOOLS_DIR}/uefi-build.sh -b ${BUILD_OPTION} -a ../arm-trusted-firmware hikey960# Generate l-loader.bincd ${BUILD_PATH}/l-loader
ln -sf ${EDK2_OUTPUT_DIR}/FV/bl1.bin
ln -sf ${EDK2_OUTPUT_DIR}/FV/fip.bin
ln -sf ${EDK2_OUTPUT_DIR}/FV/BL33_AP_UEFI.fdpython gen_loader_hikey960.py -o l-loader.bin — img_bl1=bl1.bin — 
img_ns_bl1u=BL33_AP_UEFI.fd# Generate partition table (You can download from net directly)
# See generate_ptable.sh to config your own ptablePTABLE=linux-32g SECTOR_SIZE=4096 SGDISK=./sgdisk bash -x generate_ptable.sh

Serial and flashing

To flash the loader you need to have the board in Recovery Mode (look at the following table for reference):

HiKey960 modes

Unfortunately you can’t access the Boot Screen usingminicom. For that purpose you can try ser2net(or alternatively picocombut I had more success with the former).

sudo apt-get install ser2net

Append the line for serial-over-USB in the config file etc/ser2net.conf

2001:telnet:0:/dev/ttyUSB0:115200 8DATABITS NONE 1STOPBIT banner

Boot UEFI in recovery mode. Create a connection with your Serial to USB cable (usually on either/dev/ttyUSB0{1} . You can always check this by runningdmesg before and after inserting the Serial to USB table.
Install the HiKey960 recovery tools:

git clone https://github.com/96boards-hikey/tools-images-hikey960cd tools-images-hikey960# Ensure that the l-loader is set up 
# on the correct addressscat config./hisi-sec_usb_xloader.img 0x00020000
./hisi-sec_uce_boot.img 0x6A908000
./recovery.bin 0x1AC00000

If you are using a Debian-based distribution remove the modemmanager package. This package may cause hikey_idt tool to fail.
sudo apt-get purge modemmanager
Run the command to download l-loader.bin into HiKey960, using the right USB port:
sudo ./hikey_idt -c config -p /dev/ttyUSB0{1}
Now that UEFI is running on the Recovery Mode it’s time to flash update the images!

# Flash the images onto HiKey960
sudo fastboot flash ptable prm_ptable.img
sudo fastboot flash xloader hisi-sec_xloader.img
sudo fastboot flash fastboot l-loader.bin
sudo fastboot flash fip fip.bin
sudo fastboot flash boot boot.img
sudo fastboot flash cache cache.img
sudo fastboot flash system system.img
sudo fastboot flash userdata userdata.img

Boot UEFI in Fastboot mode (see above)
Runtelnet localhost 2001
Using the Esc key, the boot manager is loaded. From there you can access UEFI Console, Android Fastboot or Grub!

Good job!!! 🥂🎇

Using U-Boot as an EFI payload

Das U-Boot is not just *another bootloader. *It’s one of the most used bootloaders in the embedded world. A brain child of Wolfgang Denk, it has been adopted by many developers and companies and they all contribute to its source base.

For a bootloader to be successful around a variety of processors and architectures, a method of configuring it is necessary. This is happening during compile time (as with the Linux kernel). This technique is significantly reducing the complexity of the bootloader.

In the case of Das U-Boot, this configuration is driven by a single header file specific to the target platform and a few soft links in the source tree, that depending on the architecture, target board, etc select the required sub-directories.

The first step of configuring Das U-Boot for a specific platform/board/architecture is done by issuing:

make <platform>_config

Configuration options are selected using macros in the form of CONFIG_XXXX. Configuration settings are selected using macros in the form of CFG_XXXX. In general, configuration options (CONFIG_XXX) are user configurable and enable specific U-Boot operational features. Configuration settings (CFG_XXX) are usually hardware specific and require detailed knowledge of the underlying processor and/or hardware platform.

Board-specific U-Boot configuration is driven by a header file dedicated to that specific platform that contains configuration options and settings appropriate for the underlying platform. The U-Boot source tree includes a directory where these board-specific configuration header files reside, namely ../include/configsdirectory where you can change anything suitable for the board you are using/porting.

For a more insider’s approach have a look at the great paper UEFI on Top of U-Boot, written from SUSE’s Andreas Färber and Alexander Graf!

Note: If I have some extra time this month, I will try to attempt porting U-Boot for the HiKey960 although I can’ t promise anything :)

Keep tuned for my approach and the issues I had on my way; my next article is out next week!

Fun Stuff

Would you like to learn more about Operating Systems, their internals and do some small (but exciting) experiments on Raspberry Pi?

I have two great suggestions for you. One is a (quite old) lab from the University of Cambridge in U.K., namely Operating Systems Development.

The other one, is from the University of Stanford and it’s very recent and relevant Writing an OS in Rust!

I hope that you can get a bit more insight from those and enjoy as much as I did.

Next steps

To conclude, I am trying to get into the habit of writing more often blogs but it feels currently that my time is better spent experimenting and keeping everybody updated with a longer (and hopefully more detailed) monthly blog entry!

Having a regular weekly chat with @microcolonel is very helpful. His kind, patient and helpful approach gives me a lot to think about, experiment and learn.

The community of Redox is very engaging and active and its future looks very bright to me!