Building Linux From Scratch on a Google Cloud Virtual Machine

Part 4: System Configuration and Kernel Popping

8 min readAug 29, 2020

Continued from Part 3

References are made to LFS Version 9.1-systemd using the format LFS x.x.x… for specific sections. Command blocks are largely copied verbatim with minor customizations for this setup.

This part of the tutorial will cover LFS sections 7 and 8. This is where we get to compile and boot into a freshly popped kernel. I’ve been waiting for this moment.

LFS 6.80 specifies an updated command for chrooting into the system after building all of the packages. I will use that to dive back in.

chroot "$LFS" /usr/bin/env -i          \
    HOME=/root TERM="$TERM"            \
    PS1='(lfs chroot) \u:\w\$ '        \
    PATH=/bin:/usr/bin:/sbin:/usr/sbin \
    /bin/bash --login

Network Configuration

We will skip this section for now and save network configuration for another day. My thought is to clone the /etc/resolv.conf of the host VM when we have a standalone, bootable build. The /etc/resolv.conf from my host VM is shown below.

System Clock

Output from hwclock --localtime --show indicates the system is set to UTC time. LFS 7.5 provides the means to set the system clock to the desired time.

Linux Console Config

No /etc/vconsole.conf is present in the Debian host system, so we will skip this for now.

System Locale

Similarly no /etc/locale.conf is present.

Inputrc

We will use the /etc/inputrc from the Debian system.

cat > /etc/inputrc << "EOF"
# /etc/inputrc - global inputrc for libreadline
# See readline(3readline) and `info rluserman' for more information.# Be 8 bit clean.
set input-meta on
set output-meta on# To allow the use of 8bit-characters like the german umlauts, uncomment
# the line below. However this makes the meta key not work as a meta key,
# which is annoying to those which don't need to type in 8-bit characters.# set convert-meta off# try to enable the application keypad when it is called.  Some systems
# need this to enable the arrow keys.
# set enable-keypad on# see /usr/share/doc/bash/inputrc.arrows for other codes of arrow keys# do not bell on tab-completion
# set bell-style none
# set bell-style visible# some defaults / modifications for the emacs mode
$if mode=emacs# allow the use of the Home/End keys
"\e[1~": beginning-of-line
"\e[4~": end-of-line# allow the use of the Delete/Insert keys
"\e[3~": delete-char
"\e[2~": quoted-insert# mappings for "page up" and "page down" to step to the beginning/end
# of the history
# "\e[5~": beginning-of-history
# "\e[6~": end-of-history# alternate mappings for "page up" and "page down" to search the history
# "\e[5~": history-search-backward
# "\e[6~": history-search-forward# mappings for Ctrl-left-arrow and Ctrl-right-arrow for word moving
"\e[1;5C": forward-word
"\e[1;5D": backward-word
"\e[5C": forward-word
"\e[5D": backward-word
"\e\e[C": forward-word
"\e\e[D": backward-word$if term=rxvt
"\e[7~": beginning-of-line
"\e[8~": end-of-line
"\eOc": forward-word
"\eOd": backward-word
$endif# for non RH/Debian xterm, can't hurt for RH/Debian xterm
# "\eOH": beginning-of-line
# "\eOF": end-of-line# for freebsd console
# "\e[H": beginning-of-line
# "\e[F": end-of-line$endif
EOF

Shells

From LFS 7.9

cat > /etc/shells << "EOF"
# Begin /etc/shells

/bin/sh
/bin/bash

# End /etc/shells
EOF

Systemconf

We will use the default settings for now.

Fstab

These commands, similar to what was used in Part 1 (see edit there), will populate the /etc/fstab file. Note that the mount point has changed and no longer includes the $LFS prefix.

echo UUID=`blkid -s UUID -o value /dev/sdb2` / ext4 discard,defaults,nofail 0 2 | tee -a /etc/fstab

Here be dragons! There are significant departures from the LFS docs needed to make this work.

Compiling the Kernel

Google cloud documentation indicates some specific configurations are needed when building a custom OS, but it took me nearly a week to figure out all of the necessary options needed to build a bootable kernel.

I ended up starting with a copy of the host system kernel config file available at /boot/config-4.19.0–10-cloud-amd64, and then I manually edited the options required per the documentation above.

Since we are building a much newer kernel in LFS, 5.5.3, new options are available that aren’t covered in the copied config file, and old options, e.g., CONFIG_KVM_CLOCK=y are gone. Running make oldconfig using the modified config file leads to a number of prompts for each of the new option. On my first pass, I used the recommended defaults, but hours of trial and error revealed some input device settings (e.g. mouse controls) and certificate settings were hindering proper booting, and those needed to be tweaked.

You can find my fully tweaked config file here. Hopefully it will work out of the box for you.

Remove any config files (including .config, .config.OLD, etc.) in the linux-5.5.3 source directory and create a fresh .config file copied from the one linked above, then proceed as follows.


make -j8 oldconfig #no warnings should be thrownmake -j8 #grab a coffee, even with 8 procs this takes some timemake modules_install

We need to copy the kernel image and associated files to the boot partition. LFS has some confusing directions here. We are going to use the hosts boot system, but we are going to keep the kernel and associated files in the /boot directory in the LFS file system (i.e., on the sdb2 partition with the rest of our compiled code). We will point the grub bootloader on the host system to the appropriate location of these files.

cp -iv arch/x86_64/boot/bzImage /boot/vmlinuz-5.5.3 cp -iv System.map /boot/System.map-5.5.3cp -iv .config /boot/config-5.5.3

! IMPORTANT ! At this point, go to LFS 9.1 and follow the directions to create the release files. I did not do this and cost me a day of troubleshooting. There’s no indication that they are critical for booting, but I could not get my distro to boot without them.

SKIP the GRUB installation (LFS 8.4)

Google cloud uses some tricky boot voodoo with EFI and GRUB. Luckily we can hijack the bootloader on the host partition to do our bidding. We will configure that after a few more steps.

Create an an initramfs

This step is absent in the LFS documentation, because it’s probably unnecessary when building LFS on actual hardware or an emulator. But it is a critical step for booting into LFS on a Cloud VM.

If you look in the /boot directory on the host system, you will see 4 related files: the kernel, the config file, the system map, and an initrd file. The initial ram disk (initrd, but it’s acutally an initramfs — initial ram file system) file is a temporary file system that is loaded into RAM at boot that helps the kernel do its thing (shame on me, I need to learn more about it).

We need to create a an initramfs for the LFS file system we’ve built. Fortunately a wonderful tool from the folks at RedHat exists to help usaccomplish this. Dracut makes it trivial to create an initramfs.

We’ll need to download the source code for dracut and cpio, a core unix tool. Since the LFS chroot environment lacks internet functionality, we’ll logout of the LFS environment and use the host system’s tools to fetch the source code and then copy back into the LFS environment.

cpio

The beyond linux from scratch supplement has directions on how to build this utility. Grab the source code using wget on the host’s machin and then copy it into the $LFS/sources directory. Chroot back into the LFS system and follow the standard build directions for cpio.

dracut

Back in the host system, get the dracut source from:

https://mirrors.edge.kernel.org/pub/linux/utils/boot/dracut/dracut-050.tar.xz

Transfer it to the $LFS/sources directory, chroot back in, untar and enter the dracut source directory.

./configuremakemake install

Navigate to /lib/modules. This directory contains a directory of the kernel modules for the kernel version you compiled. The directory should be called ‘5.5.3’. Point dracut to that directory and pass a filename for the output file (the initramfs).

Make the initrd file

dracut initrd.img-5.5.3 5.5.3

That created the initrd.img-5.5.3 file in the /lib/modules directory. Move that file to /boot, which should now have 4 files in it: the kernel, config, map, and initrd files.

Configure the bootloader on the host system.

Exit the LFS system to the host system. Complete the following steps as root.

#Ensure a device map exists in the host boot directory
grub-mkdevicemap#Turn off os-prober
sed -i \ 's/GRUB_DISABLE_OS_PROBER=false/GRUB_DISABLE_OS_PROBER=true/g'

Add a boot entry for the LFS system.

Edit /etc/grub.d/40_custom to contain the following:

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'NewLFS' {
        insmod part_gpt
        insmod ext2
        set root='hd1,gpt2'
        echo    'Loading Linux 5.5.3 ...'
        linux   /boot/vmlinuz-5.5.3 root=UUID=[YOUR-UUID-FOR-LFS-PARTITION] ro console=tty0 console=ttyS0,115200
        echo    'Loading initial ramdisk ...'
        initrd  /boot/initrd.img-5.5.3
}

This adds an entry to the GRUB menu that points the VM to look for an OS on /dev/sdb2 (marked as ‘hd1/gpt2’ in GRUB parlance). It specifies that the root filesystem is on the /dev/sdb2 partition (marked with theUUID), and that the kernel and initrd can be found in the /boot directory .

Update GRUB with