Getting your feet wet with Ubuntu Autoinstall

9 min readMay 12, 2024

As sometimes happens this entire project started by accident. I was merely installing Ubuntu on a newly arrived rack workstation. And all of a sudden, a no-brainer task transformed into a challenge that eventually blew my mind and got me onto a new level.

Since it’s not a real server box the machine has no IPMI on board. This fancy piece of iron carries 2 USB-C/DisplayPort for monitor connectivity instead of conventional display interface sockets. Unfortunately, it does not come with a branded USB-C/DP — HDMI or DP adapter out of the box. The one I had in the office at hand was a cheap no-name that required serious witchcraft to get to see something on the screen until I could SSH into the box.

As this machine was primarily intended for testing, I had to reinstall the OS quite a few times. This quickly led me to a question: what if there was an alternative to all this hassle with the monitor? What if there was a way for a completely headless unattended setup? Since it doesn’t have an IPMI interface the short answer is no. You need to choose your boot media at least or better configure the BIOS boot options. I should note that not only it wouldn’t boot from external media by default. It also requires a RAID setting to get it to see its own internal SSD. All this with a barely working monitor as we remember. The good news is you need to do that only once.

Once you are past that, things change. Googling around got me to the Ubuntu autoinstall https://ubuntu.com/server/docs/install/autoinstall Unfortunately, it’s not well documented and can be misleading at times. This has led me to a lot of confusion with cloud-init and cost quite a detour until I understood the differences. Will explain the details later.

On the other hand, that path has also gained me valuable knowledge that proved very useful in the next episodes. My article series is an attempt to bridge the gap in documentation and spread the word on this cool Ubuntu feature and interesting use cases we found for it.

The first article is about getting your feet wet starting with the simplest scenario possible. We are going to create a custom bootable ISO that installs a minimal viable Ubuntu server. All settings default, DHCP for network, OpenSSH and a user with a password. I’m going for a password SSH authentication, but you can replace it with a key.

All of it headless and fully automagically. No monitor, no keyboard, just insert the USB drive, press the power button and get a system you can SSH into. And yes, it really does work this way.

Getting the moving parts.

Autoinstall uses a yaml config stored in a file that should be called “user-data”. For a self-contained ISO we will store it in a folder called “/nocloud”. So here goes /nocloud/user-data:

#cloud-config
autoinstall:
  version: 1
  identity:
     hostname: ubuntu22auto
     username: ubuntulogin
     password: <mkpasswd -m sha-512 ubuntupass>
  ssh:
     install-server: yes
     allow-pw: yes

This is the minimal viable config for this job.

There’s one more thing to consider. Once finished, your machine will need to reboot. Since our installer is fully unattended, it will neither prompt for anything, nor will there be any indication. My favorite joke about this:

A man comes to a patent office to register his invention — a universal shaving machine. The patent clerk asks:

- Could you please explain how your machine works?

- Well, you insert your face, press a button and get a shave.

- Ok, how about people’s faces being different?

- Indeed, they are, but only on the first run.

Some hypervisors and IPMI cards would disconnect the installation media upon reboot. But let’s play it safe. The cleanest option for a headless setup in my opinion is to shut down the instance upon the installation. This is going to be a clear indication to remove the installation media and power on your system. You can configure it by including a bash command (we will cover that later) but autoinstall already has a built-in function. Just add:

  shutdown: poweroff

Ok, now that we’ve got the autoinstall yaml we need a way for Ubuntu installer to engage it. This is configured in grub.cfg:

menuentry “Ubuntu autoinstall test” {
 set fgxpayload=keep
 linux /casper/vmlinuz “ds=nocloud;s=/cdrom/nocloud/” debug autoinstall ---
 initrd /casper/initrd.gz
}

This is what really triggers it: “ds=nocloud;s=/cdrom/nocloud/” debug autoinstall. This is the folder where the autoinstall user-data yaml file is stored as I mentioned before. It did cost me quite a lot of gray hair to get it working — the quotes did the trick for me. But you’re welcome to experiment with the escape characters to run without the quotes. You can also replace ‘debug’ with ‘quiet’ for the production builds.

Setting up a build environment.

Ok, our configuration is ready but how do we inject it into an Ubuntu ISO to get a working image?

Autoinstall documentation usually describes the hard way and in a limited fashion. We’ll get to that in the later articles as we get to the advanced scenarios and building our images with a CI/CD pipeline. But let’s start with the easy way and let the complexity evolve gradually.

The quick and easy way is an application called CUBIC, Custom Ubuntu ISO Creator, https://github.com/PJ-Singh-001/Cubic. Let’s set it up. I will spin up an empty Ubuntu 22.04 Desktop VM as our build machine. The rest is straightforward. Simply follow the steps from the readme guide:

sudo apt-add-repository universe
sudo apt-add-repository ppa:cubic-wizard/release
sudo apt update
sudo apt install --no-install-recommends cubic

Next type “cubic” in the terminal to launch and create a project folder. I opted for U22AutoInstallBuildV0.1:

Next let’s grab a vanilla Ubuntu server image. I will download it into our Downloads folder. Please note the link I use, it will be important later:

wget http://releases.ubuntu.com/22.04/ubuntu-22.04.4-live-server-amd64.iso

Once it’s downloaded, let’s select our base image and configure the project. CUBIC is going to pull metadata from the ISO. Let’s customize it:

Unpacking the source image:

Now we enter the chroot shell where we can customize the system. We will do it in the later articles of the series but let’s skip it for now:

Here we could customize the kernel version if there were multiple to choose from. The “Preseed” tab looks like legacy Debian/Ubuntu distros to me. Current Ubuntu installer is Subiquity and it does not use preseed files:

Customizing the build with VSCode.

Now the “Boot” tab is useful. Remember we need to edit the grub.cfg?

You can do that here but I prefer VSCode.

If you don’t have the OpenSSH server installed on the VM, run:

sudo apt install openssh-server

Next, copy the ssh key into the VM, so that VSCode does not drive you crazy asking for password 100 times. On Windows I use MS Windows Terminal you can install from the Microsoft Store. It has the ssh client built in:

ssh-keygen

The resulting id_rsa & id_rsa.pub keys are stored in .ssh folder within your user home directory. Works similar on other operating systems.

If you work with a clean Ubuntu, please make sure there’s at least an empty ~/.ssh/authorized_keys file in your home directory. If not, let’s create it:

mkdir -p ~/.ssh
cd ~/.ssh
touch authorized_keys

Now we can copy the public ssh key from our main machine. On Windows run:

scp ~/.ssh/id_rsa.pub ubuntulogin@<remotehost>:~/.ssh/authorized_keys

Now let’s test:

ssh ubuntulogin@<remotehost>

Now let’s switch into the VSCode and connect. Please make sure you have the Remote extension pack or at least Remote-SSH installed:

Don’t forget to change the username:

Next it will prompt you for the config and launch a new window for our remote session. I will open the project folder directly to avoid any distraction:

This is the structure of Ubuntu distro CUBIC unpacked for us. We will examine this in more detail in the later articles. For now all we care about is the custom-disk folder:

First create the nocloud folder and the user-data yaml that we drafted above. I will also add an empty file called meta-data. We’re not going to use it but a stub is required.

Another important item is our user password:

mkpasswd -m sha-512 ubuntupass

results in this hash, that needs to be copied into the config file:

$6$LmPUjxOfMHOMgRlg$pSXXVlcfwSKUSotcoG6ed7DUu7.iOX7kJEylN9V9z3C96uNBIMfCJjtL1tNjLx9dDbKS/kH9W7B8oIMKxmXb70

If mkpasswd is not installed by default

install it as required:

sudo apt install whois

Now this is a working user-data yaml:

We’re almost set, just need to edit the grub.cfg. The file is stored in the custom-disk/boot/grub directory:

It’s a basic config. I added a new menu entry and it will boot automatically by default as it’s the first on the list.

Another change is the timeout that I changed from 30 seconds to 10 but could be less if you don’t need an interaction.

Compiling the ISO.

That’s it. Now we’re ready to compile the ISO. Press “Next” and we’re ready to go. I personally always use xz but will skip the explanation for now.

CUBIC selecting the ISO compression method

Testing the image.

CUBIC also offers a “Test” feature to launch the ISO using QEMU. It could be useful for some basic scenarios (make sure you have the virtualization nesting enabled). Let’s give it a try:

However, in my experience it proved unstable, especially since I normally run these tasks inside VMs:

Your mileage may vary, depending on the hypervisor of choice, resources you allocate, some tweaks, moon phase, etc. ESXi would often work for my simple cases but I’m running this particular VM on Hyper-V. Anyway, for complex real-life builds I ended up ditching it in favor of regular VMs.

Let’s take a dedicated VM for our test, make sure you got the standard setting for Ubuntu right. ESXi, Proxmox, XCP-NG, LXD do not require any unusual tweaks. Hyper-V used for this demo requires disabling “Enable Secure Boot”:

Hyper-V requires unchecking “Enable Secure Boot”.

Next attach our ISO…. and it just works:

Summary.

In this article we have learned building a very basic automatically installed Ubuntu image. I have also started to show some caveats on the way. We have set up a dev environment with all the necessary tooling and tested the build.

Stay tuned for the cool stuff coming.

Getting your feet wet with Ubuntu Autoinstall

Written by Boris Kaplun