Building Linux From Scratch on a Google Cloud Virtual Machine
Part 1: The Build Environment
In Part 1 of this series, we will set up a build environment to build Linux From Scratch.
Links to Part 2, Part 3, Part 4
I decided to build Linux from scratch as a learning project to gain a better understanding of Linux internals. I spend a lot of time developing in the cloud, and cloud technology seems to have Linux at its heart, so it seemed worthwhile to give this a shot. Besides, I have major graybeard yearnings.
Note: we will be following LFS Version 9.1-systemd. I will use “LFS x.x.x…” when referring to specific parts. Many of the commands are copied verbatim.
Some of my specific learning goals:
- source compilation and debugging — I’m C-illiterate
- increased use of binutils and coreutils
- low-level kernel features like namespaces — what make containers tick
Building a new Linux system is done most easily within an existing Linux system. The easiest way to get up and running with a Linux system is to fire up a virtual machine on one of the many cloud services. I use Google Cloud Platform at my job, so we will use it here since I’m familiar with it.
Start a GCP VM Instance
1. Sign up for the free tier of Google Cloud Platform and create a new project. Google’s documentation is excellent, so if you’re stuck somewhere, just google it.
I will use gcloud commands in this series for consistency because sharing text commands is clearer than point and click descriptions. Anything done with these commands can also be done with the Cloud Console GUI.
2. Install the gcloud SDK. It is a command line utility that makes interacting with Google Cloud facilities easy. Follow the documentation to authenticate and set the configuration.
Here’s the output gcloud config list for my configuration:
3. Create a VM with the following command. Replace the instance name, zone, and disk name accordingly.
That command did a number of important things:
- Provisioned a 1-cpu, 3.75GB VM running Debian Linux. We can (and will) increase the compute resources when we start compiling source code.
- Enabled a serial port that will let us interact with the VM during boot time. This will be critical when we want to switch between Linux kernel installations. You can read more about serial ports and consoles here.
- Created and attached a secondary persistent disk. We will need to do some disk and filesystem manipulation throughout the course of the build, and it doesn’t seem possible to partition the primary boot disk without borking the VM. Using a secondary disk is a way around this. (GCP sets 200GB as the default minimum for performance reasons)
4. Connect to the VM using gcloud.
Use this command to shell into the Linux os:
gcloud compute ssh [INSTANCE-NAME]
This gcloud feature is very convenient. It saves us from having to manually configure ssh and transfer keys.
Set up a user password. It could save us a headache down the road.
sudo passwd $(whoami)
Update the package repos and upgrade packages.
sudo apt-get update
In another terminal, use this command to open a serial console:
gcloud compute connect-to-serial-port [INSTANCE-NAME]
Notice the output of the serial console is different from the shell. It displays hardware and system-level diagnostic information. We will use it primarily to interact with the bootloader and to get ourselves out of any goofs should we make them.
The default system configuration has the bootloader timeout set to 0s, which means we can’t interact with it. Let’s change that and then reboot so we can see the bootloader menu in the serial console.
In the shell session, edit /etc/default/grub.d/15_timeout.cfg. Set the timeout to 30s. Update GRUB(the bootloader) with sudo update-grub and then reboot the VM with sudo reboot .
Monitor the serial console. You should see something like this:
That’s the GRUB bootloader menu, and access to it will be key in later steps.
Feel free to keep the serial console open, but reconnect a shell session for the next part.
Format and Partition the Attached Disk
EDIT (8–27–20): After progressing through the final stages of building, it became clear that the boot and home partitions are not needed. We will use the boot partition on the primary disk (sda). A primary ‘root’ partition on sdb is still needed.
The secondary disk we attached needs to formatted, partitioned, and mounted before we can do anything with it.
- Find your secondary disk with the
lsblkcommand.
You should see a tree structure with the primary disk, sda, and its partitions, and the secondary disk, sdb, which has no partitions.
2. The disk needs to be labeled before it is recognized by the system and bootloader. We will use the parted utility to label sdb as gpt (GUID Partition Table)
sudo parted /dev/sdb mklabel gptYou might see an informational warning about updating /etc/fstab. We will address that later.
We need to partition sdb before we start working on it. LFS 2.4 recommends a main or ‘root’ partition and two additional partitions: boot and home. Our disk is 200GB, so will assign 4GB to boot, 96 GB to home, and 100GB to main.
3. Create the partitions.
#boot
sudo parted -a opt /dev/sdb mkpart primary ext4 0% 2%#primary
sudo parted -a opt /dev/sdb mkpart primary ext4 2% 52%#home
sudo parted -a opt /dev/sdb mkpart primary ext4 52% 100%
The output of lsblk should look like this:
4. Make file systems on the partitions.
#boot
sudo mkfs.ext4 -L datapartition /dev/sdb1#main
sudo mkfs.ext4 -L datapartition /dev/sdb2#home
sudo mkfs.ext4 -L datapartition /dev/sdb3
5. Set the LFS environment variable.
This points to the mount point for the LFS installation. We will need to set it for both user and root on the build system.
#user (i.e., your current login)
cd ~
echo 'export LFS=/mnt/lfs' >> .bashrc
source .bashrc#root
sudo -s #start a root session
cd /root
echo 'export LFS=/mnt/lfs' >> .bashrc
source .bashrc
exit #leave root session#test environment variable -- should return '/mnt/lfs'
echo $LFS
6. Mount the partitions.
#make the mount directories
sudo mkdir $LFS
sudo mkdir $LFS/boot
sudo mkdir $LFS/home#mount 'root' partition
sudo mount -v -t ext4 /dev/sdb2 $LFS#mount boot partition
sudo mount -v -t ext4 /dev/sdb1 $LFS/boot#mount home partition
sudo mount -v -t ext4 /dev/sdb3 $LFS/home
The output of lsblk should look like this:
7. Update /etc/fstab so the system mounts the partitions automatically at boot
echo UUID=`sudo blkid -s UUID -o value /dev/sdb2` $LFS ext4 discard,defaults,nofail 0 2 | sudo tee -a /etc/fstabecho UUID=`sudo blkid -s UUID -o value /dev/sdb1` $LFS/boot ext4 discard,defaults,nofail 0 2 | sudo tee -a /etc/fstabecho UUID=`sudo blkid -s UUID -o value /dev/sdb3` $LFS/home ext4 discard,defaults,nofail 0 2 | sudo tee -a /etc/fstab
Prepare the host system requirements (build tools).
- Run the script provided in LFS 2.2 to check if all the needed build tools are installed.
Your output should be similar to this:
bash, version 5.0.3(1)-release
/bin/sh -> /bin/dash
ERROR: /bin/sh does not point to bash
Binutils: version-check.sh: line 10: ld: command not found
version-check.sh: line 11: bison: command not found
yacc not found
bzip2, Version 1.0.6, 6-Sept-2010.
Coreutils: 8.30
diff (GNU diffutils) 3.7
find (GNU findutils) 4.6.0.225-235f
version-check.sh: line 25: gawk: command not found
/usr/bin/awk -> /usr/bin/mawk
version-check.sh: line 35: gcc: command not found
version-check.sh: line 36: g++: command not found
(Debian GLIBC 2.28-10) 2.28
grep (GNU grep) 3.3
gzip 1.9
Linux version 4.19.0-10-cloud-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.132-1 (2020-07-24)
version-check.sh: line 41: m4: command not found
version-check.sh: line 42: make: command not found
version-check.sh: line 43: patch: command not found
Perl version='5.28.1';
Python 3.7.3
sed (GNU sed) 4.7
tar (GNU tar) 1.30
version-check.sh: line 48: makeinfo: command not found
xz (XZ Utils) 5.2.4
version-check.sh: line 51: g++: command not found
g++ compilation failedNote that some packages are not found. We will have to install them. For those packages that are installed, a version is listed. We need to ensure that the version meets or exceeds those specified in LFS 2.2. Since we are working on an up-to-date Debian system, the versions all exceed the recommendations.
The missing packages can be installed with:
sudo apt-get install build-essential, bison, gawk, texinfo2. Download the source files needed to build Linux from scratch.
LFS 3.1 provides a manifest of source file download URIs that can be used with wget. You might need to install it first: sudo apt-get install wget.
Create the source directory for the files and modify its file permissions.
sudo mkdir -v $LFS/sources
sudo chmod -v a+wt $LFS/sourcesObtain the package manifest ‘wget-list’.
wget http://www.linuxfromscratch.org/lfs/view/stable-systemd/wget-listThe file ‘wget-list’ should be in your home directory. Now download the source files to the source directory.
sudo wget --input-file=wget-list --continue --directory-prefix=$LFS/sourcesTo check that everything was downloaded, we can make a list of all the packages in the wget-list and compare it to the contents of the sources directory. One of my downloads timed out on the first pass and was missing.
#create a the list of expected file names
grep -oP “([^\/]+$)” wget-list | sort -f > manifest#create the list of downloaded file names
ls /mnt/lfs/sources | sort -f > downloaded#compare the lists
diff manifest downloaded
3. Make the tools directory and symlink it to the host system.
sudo mkdir -v $LFS/toolssudo ln -sv $LFS/tools /
The reason for the symlink as directly described in LFS 4.2:
The created symlink enables the toolchain to be compiled so that it always refers to
/tools, meaning that the compiler, assembler, and linker will work both in Chapter 5 (when we are still using some tools from the host) and in the next (when we are “chrooted” to the LFS partition).
4. Set up the lfs user.
sudo groupadd lfssudo useradd -s /bin/bash -g lfs -m -k /dev/null lfssudo passwd lfs #set passwordsudo chown -v lfs $LFS/toolssudo chown -v lfs $LFS/sources#switch to lfs login shell
su - lfs
5. Continue the environment set up as described in LFS 4.4
This completes Part 1.
