Network Booting an IBM Cloud Bare Metal Server for VPC

Bare Metal Servers for VPC allow you to provision a plethora of images — from Windows, to VMWare ESXi, to various Linux distributions. But what if IBM Cloud doesn’t have quite the right image? As long as you are deploying one of the supported Operating Systems, you can upload custom images based on that operating system. But:

• What if you want to install an OS we don’t currently have an image for?
• Or what if you just want to control the install using your tried and true PXE infrastructure or to do things such as encrypting your boot drive?

Well — read this post, and you’ll be ready to deploy a bare metal server using iPXE over HTTP.

Solution Overview

• Deploy an image that contains an iPXE EFI executable that is executed as the first stage bootloader
• Provide an iPXE script within your image that loads an iPXE script from a remote PXE server over HTTP/S.

High Level iPXE Solution on Bare Metal for VPC

Once the custom image is deployed, the boot flow is as follows, where the last step is loading a script off of your PXE server which then gives you full control over how you want to install your operating system.

Boot Flow

Building the iPXE EFI

First we need to build the iPXE EFI binary with the embedded iPXE script. This is fairly straightforward, but here is a quick and dirty solution using a docker container:

# Run a rockylinux docker container mounting your current working directory
docker run -v $(pwd):/work -it rockylinux:9 bash
# Install needed tools
dnf install gcc make git -y
# Clone the ipxe git repo
git clone https://github.com/ipxe/ipxe.git

# Create the boot.ipxe embedded script
cat <<EOF > boot.ipxe
#!ipxe
chain file:script.ipxe || echo "Unable to load script.ipxe in /EFI/BOOT/"
shell
EOF

# Create the ipxe.efi with the embedded boot.ipxe script
cd ipxe/src
make bin-x86_64-efi/ipxe.efi EMBED=../../boot.ipxe

# Copy the ipxe.efi to your mounted filesystem
cp ipxe.efi /work/

# Exit your docker container
exit

Building the iPXE Custom Image

The image needs to contain an EFI boot partition with two files:

• /EFI/BOOT/BOOTX64.efi (this is your ipxe.efi)
• /EFI/BOOT/script.ipxe (this is your ipxe script that points to your iPXE server)

But, to successfully deploy the image on Bare Metal for VPC today, you need to use a currently supported base image. To do this, we’ll use Debian 11 as our base as it’s small and free. From a Linux host (I used an Ubuntu Virtual Server Instance that was also hosting my iPXE server), create the image as follows:

# Get a vanilla debian image
get https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-amd64.qcow2
# Enable qemu-nbd to be able to manipulate the contents of the qcow2 image
modprobe nbd max_part=8
# Connect the qcow2 to /dev/nbd0 - use this as your block device
qemu-nbd --connect=/dev/nbd0 debian-11-generic-amd64.qcow2
# Remap partitions and make them available in your /dev/ filesystem
sgdisk -sg /dev/nbd0
# Mount the EFI boot partition
mount /dev/nbd0p2 /mnt/
# Copy the iPXE efi binary (ceated earlier) and name it to allow
# it to be the default boot binary
cp ~/ipxe.efi /mnt/EFI/BOOT/BOOTX64.EFI

# Create an iPXE script that sets up the network and the appropriate URL to chainload
# NOTE:  The IP:PORT of the URL must be correct. You can use DNS as well.
cat <<EOF > /mnt/EFI/BOOT/script.ipxe
#!ipxe
dhcp
chain http://10.240.128.4:8080/script.ipxe || shell
EOF

# Clean up other EFI files to prevent them from loading
rm /mnt/EFI/BOOT/grubx64.efi
rm -rf /mnt/EFI/debian
# Clean up
qemu-nbd --disconnect=/dev/nbd0

# Next upload your modified qcow2 image to Cloud Object Store.

Now that we have an image, we need to upload the image to our cloud object store (COS) bucket, and then import the image as a custom image based on Debian 11. You can find detailed instructions on importing custom images in the IBM Cloud Docs.

Setup your iPXE Server

There are a million ways to do this, but in short, you need:

1. A script.ipxe file
2. A webserver to serve it.

What I show below is a simple way to iPXE boot the Fedora install ISO. You can of course do a full blown automated install with kickstart, but that’s beyond the scope of this post!

# Create the script.ipxe
wget http://download.fedoraproject.org/pub/fedora/linux/releases/37/Server/x86_64/iso/Fedora-Server-netinst-x86_64-37-1.7.iso
cat <<'EOF' > script.ipxe
#!ipxe
# Use the iso image for the installer (no kickstart for this simple demo)
sanboot Fedora-Server-netinst-x86_64-37-1.7.iso
EOF

# Start up a simple webserver that binds to the VPC IP of my PXE server
ruby -run -e httpd . -b 10.240.128.4 -p 8080

Create your Bare Metal and let it loose!

Your almost there! Now just create the IBM Cloud Bare Metal Server for VPC and select the custom image you created. Sit back — your done. The server will take about 10–15 minutes to provision and start the PXE boot process. Once it hits Starting, you can open up a console and watch it for fun!

PXE Booting an IBM Cloud Bare Metal Server for VPC

Being able to PXE boot your bare metal server gives you the ultimate control over your installation process and what operating system you wish to run. It even lets you run as a diskless system if that’s your desire. Setting up the custom image is a bit cumbersome, but do it once and you can continue to use that image for as many PXE boots as your heart desires!