Android ROM Cooking

A brief navigation guide to the world of Android ROM manipulation

Android is the fastest growing OS for embedded devices that expect user interactions and cater to multimedia. A major reason for the steady adoption of Android is that it’s application framework works on top of a Linux kernel and has modest hardware expectations. In this blog I intend to pen my understanding of cooking up an Android ROM which is a fast way of creating a modified/custom version of Android OS image from an existing stable one.

System On Chip and Android

A system on a chip is an integrated circuit that integrates all or most components of a computer or other electronic system. These components almost always include a central processing unit, memory, input/output ports and secondary storage — all on a single substrate or microchip, the size of a coin — Wikipedia

The conception of a new Android hardware starts at a SoC. The SoC encapsulates the CPU(hence the architecture), memory and a host of core functionality sensors (even things like GPS and WIFI/Bluetooth components). Hence the linux kernel that Android application framework operates on is tightly coupled with the SoC. An OEM then takes this SoC and produces the final android based device. Motorola is an OEM, Qualcomm’s Snapdragon is a series of SoC and Moto phones in this case are the end product. There are a whole lot of components like display, camera, fingerprint readers purely dictated by the OEM. These contribute to additional linux drivers required.

From the above description I intend to highlight the fact that although Android is an open source project a whole lot of the source code that runs the end device you use is dictated by a host of third parties like SoC manufacturers and the OEM. Thus, you cannot just build the OS image required for your device using the AOSP repository.

ROM Cooking Scope

In my opinion ROM Cooking is an art :) (Yes I love calling myself a tech artist). The scope of changes that can be made to the ROM is inexact. It heavily depends on the end hardware device and how open source its software and hardware components are. It also depends on how well versed you are with a particular release (Nougat, Oreo, Pie etc) of AOSP. Android keeps shifting towards more opinionated framework and more user driven permission model to enhance security resulting in frequent deprecation of features.

In general upgrading system libraries like moving to a new version of USB driver or a more efficient video library, drop in replacement of apps like Launcher, escalating privileges of certain apps (root user access), changing boot video and animation are some very common use cases you can achieve with ROM cooking. The value add is that you can do all these without access to source code from OEM or SoC manufacturer and yet create a significant contribution to device user ‘s experience.

In this blog my intention is to highlight all the aspects of Android OS that an enthusiast should know in my opinion as they start getting their hands dirty. As for actual ROM manipulation I leave it to the artist :).

Android File System and Partitions

Android devices storage is predominantly flash memory. The knowledge of file system is important as it dictates the final OS image generated post ROM cooking. You would be “flashing” this image to the Flash Memory chip on the device and hence it is critical to ensure that it follows the file system format expected by the kernel code.

Android storage hierarchy in general has six main partitions boot, system, recovery, data, cache, and misc. You can read about the relevance of each at a high level here. Boot, system and recovery are of key importance from point of view of ROM manipulation and I will address that in following sections.

Fastboot Magic

In Android, fastboot is a special diagnostic and engineering protocol that you can boot your Android device into. While in fastboot, you can modify the file system images from a computer over a USB connection. — Android Central

To understand the significance of fastboot you need to go way down the hardware hierarchy to micro-controllers like Arduino and ESP32. These tiny bits that power a lot of hobby projects also work using flash memory. What is of significance here is that they have a mode for upload and a mode for normal run. The device can be switched to upload mode by setting a particular hardware pin to high or low. When a device is in upload mode all the data sent via UART (USB) is saved to the flash memory. These devices do not run a OS. They contain just enough code to serve minimal functionality. Thus when you mess up the code and device is locked in a reboot loop, upload mode comes to rescue as you flash your updated code.

In case of Android the pin is read by the boot-loader which pauses the boot sequence and starts listening on the UART for messages in a protocol called fastboot. You need to install the fastboot client on your workstation and you are set for a very streamlined flash memory manipulation. For instance fastboot allows you to flash a new copy of a specific partition like recovery by name.

fastboot flash recovery twrp.img

Imagine doing this by first manually figuring out the byte address of the partition and then doing a dd to that. Fastboot abstracts this making it very convenient for developers.

The hardware pin that stops the boot sequence to activate fastboot is the reason why a host of Android phones require you to press some finger breaking combination of buttons :D to enter fastboot mode.

Boot-loader and OEM locks

From the previous sections I think you can appreciate the fact that an artist can easily make some damaging/malicious changes to the Android device over fastboot. The artist may decide to overclock the CPU causing the phone to burn or swap out the ssl library to compromise the system security.

In order to protect themselves against unethical warranty claims and bad user experience, OEMs enable a lock on the boot-loader and kernel code.

The lock in short translates to “You can only flash images to boot and kernel that are approved by the OEM”. These locks are facilitated by public key cryptography. The fastboot protocol in such boot-loaders only allow verified images to be flashed. The key used to verify images is stored in the boot partition in a device tree blob.

fastboot oem unlock

OEMs provide processes to unlock the boot-loader. However, once you do it your warranty is declared void in general. Additionally, as per standard implementation of fastboot for Android all the user data gets wiped out. Hence it is wise to take a data backup of your device before setting off on this adventure.

U-Boot is a prominent embedded device bootloader and fastboot implementation is a part of its Github repository. For the state of art on secure boot in Android you can read up on Trusty.

Recovery the savior

The recovery partition of Android OS image encapsulates basic ROM manipulation functions via adb and has a minimal system image that you can fallback to when you mess things up. Recovery is also navigated to by pressing some specific key combinations on your device thus pausing the boot sequence. In this case you usually get a boot-loader menu that can take you to recovery.

Custom recoveries like TWRP make it easier to play around with ROMs as they abstract things like taking backups, flashing partitions, and enabling root user access on your device.

You usually download TWRP for your device and flash it after unlocking the ROM. If the previous sentence made sense then I think you have been a really keen reader. :)

System

The system partition contains certain high privileged apps like settings that should not really be installed from third party sources and also drivers for critical functions.

One Image to Rule them all

In the sections above we saw how we can flash individual partitions, however in general if you want to redistribute your work like Lineage OS does you would want to produce a single image that can be flashed. Fast-boot supports flashing a whole image. This is usually done by creating a flash-all script. The catch however is to know the image packing expected by your Android source code which is mostly the file system format and partition names.

You can fire something similar to the command below on your Android device adb shell to find partition names and associated block devices. The “msm_sdcc.1” part could be a different name for different devices.

ls -al /dev/block/platform/msm_sdcc.1/by-name

Bricked Devices

Playing with boot is playing with fire as you could end up messing up the boot sequence and bricking your device. If you are lucky you might be able to locate the hardware pin for enabling upload mode. This serves as the last ditch effort to revive the device. If not you have a fancy paper weight now :D.

That is all the wisdom I have come across in my experience as a young ROM artist :). I hope I made this a worthwhile read.

Looking forward to questions and comments.

Thanks!