Part 2: Enhance Packer with Ansible provisioner for Raspberry Pi

In the first article, we created custom Raspberry Pi OS images and tested them on our own machine before booting an actual Raspberry Pi hardware.

In this article, we will expand on building a custom OS image — we will do it with an Ansible provisioner

What is Ansible? Compared to a shell script, Ansible is much easier to read, write, debug and does more with much less code. An Ansible playbook is a collection of tasks, much like a shell script is collection of commands. But oftentimes Ansible playbooks will contain less tasks than commands in a shell script.

Furthermore, an Ansible playbook has a twofold purpose in our scenario:

1. used by Packer to create custom Raspberry Pi OS image
2. can still be used independently to set up new Raspberry Pi devices without Packer and update some configuration or add new functionality to existing devices.

But all that can be achieved with a shell script as well. Can’t it? What is the difference then?

The biggest difference is that in order to run a shell script on a device, it needs to be present there. Running an Ansible playbook requires no such thing. Ansible playbook can target remote hosts, even multiple at a time.

Ansible is also idempotent — by rerunning a playbook, tasks that were already run or that make no change to the current system state will simply be skipped. Only those tasks that update something or tasks there were newly added will be run.

Ansible is also a nice way to manage a fleet of devices. It stores inventory information like connection details and custom variables for each device, it has a simple to use encryption and automatic decryption tool called Vault, it uses excellent YAML syntax and so on.

So how to use Ansible with Packer? We must install it somewhere. Since we run Packer in a Docker container, we must have a Docker image that contains both Packer and Ansible installations. The simplest solution is to extend packer-builder-arm Docker image from part 1 with Ansible installation from apt. Write a Dockerfile like the following:

# syntax = docker/dockerfile:1.3

FROM mkaczanowski/packer-builder-arm

ENV DEBIAN_FRONTEND=noninteractive

RUN --mount=type=cache,target=/var/cache/apt,id=apt \
    apt update \
    && apt install software-properties-common gpg-agent --no-install-recommends -y \
    && add-apt-repository --yes --update ppa:ansible/ansible \
    && apt install ansible --no-install-recommends -y \
    && ansible-galaxy collection install ansible.posix \
    # install other Ansible roles here if needed
    && (rm -f /var/cache/apt/archives/*.deb \
    /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin /var/lib/apt/lists/* || true)

To build this Docker image simply run:

docker image build --build-arg BUILDKIT_INLINE_CACHE=1 --progress=plain -t packer-builder-arm-ansible:vlatest .

Now we can use this packer-builder-arm-ansible:vlatest Docker image to run Packer with Ansible provisioner.

We are also going to upgrade our Packer template by using HCL syntax instead of JSON. HCL is more human-friendly and can have variables and comments, unlike JSON that is more oriented towards machine parsing.

Using Ansible provisioner in HCL Packer template might look like this:

provisioner "ansible" {
  extra_arguments = [
    "--connection=chroot",
    "-e ansible_host=/tmp/rpi_chroot",
    "-e ansible_user=pi"
    ]
  playbook_file   = "setup-raspberry.yml"
}

It is important to use chroot connection and this specific ansible_host.

However, in this configuration with Packer, Ansible playbooks are run as root user. This is most likely not what you want. If we run the playbook with a non-root user, we can simply do privilege escalation only when we definitely need it. So let us fix this problem by explicitly defining become: true and become_user: pi in this setup-raspberry.yml playbook for Packer:

-
  name: Setup RPi
  hosts: all
  become: true
  become_user: pi
  tasks:
    # use only become_user: root if task should be run as superuser

    - name: Change hostname
      become_user: root
      hostname:
        name: raspberrypi-provisioned-by-packer

    - name: Create a file
      copy:
        dest: ~/test.txt
        content: "This is a test txt file"

    - name: Enable SSH
      become_user: root
      file:
        path: /boot/ssh
        state: touch

    - name: Add temporary password for pi user
      become_user: root
      user:
        name: pi
        password: "{{ 'r3notaRE' | password_hash('sha512') }}"

    - name: Enable zswap with default settings
      become_user: root
      replace:
        path: /boot/cmdline.txt
        regexp: '[^#\n]$'
        replace: ' zswap.enabled=1'

    - name: Download resize2fs_once binary
      become_user: root
      get_url:
        url: https://raw.githubusercontent.com/RPi-Distro/pi-gen/master/stage2/01-sys-tweaks/files/resize2fs_once
        dest: /etc/init.d/resize2fs_once
        mode: 0755

    - name: Force automatic rootfs expansion on first boot
      become_user: root
      systemd:
        service: resize2fs_once
        enabled: true

If ever in doubt how to write an Ansible task, you can always fallback to using a shell or command module:

- name: Command module is simple - no piping is allowed
  become_user: root
  ansible.builtin.command:
    cmd: chmod +x /etc/init.d/resize2fs_once

- name: Shell module is more complex and allows piping
  become_user: root
  ansible.builtin.shell:
    cmd: echo 'pi:r3notaRE' | chpasswd

However, Ansible will surely express how it feels about that if Ansible Lint is installed:

• chmod used in place of argument mode to file module
• Commands should not change things if nothing needs doing

Tying it all together, folder structure will look like this:

• setup-raspberry.yml - your provisioning playbook
• raspios.pkr.hcl - Packer template
• raspberry-pi.img - build result

Complete Packer template in HCL might look like this:

# reuse this long string
variable "raspios_url" {
  type    = string
  default = "https://downloads.raspberrypi.org/raspios_armhf/images/raspios_armhf-2023-02-22/2023-02-21-raspios-bullseye-armhf.img.xz"
}

source "arm" "pi" {
  file_checksum_type    = "sha256"
  file_checksum_url     = "${var.raspios_url}.sha256"
  file_target_extension = "xz"
  file_unarchive_cmd    = ["xz", "--decompress", "$ARCHIVE_PATH"]
  file_urls             = ["${var.raspios_url}"]
  image_build_method    = "resize"
  image_chroot_env      = ["PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin"]
  image_mount_path      = "/tmp/rpi_chroot"
  image_partitions {
    filesystem   = "vfat"
    mountpoint   = "/boot"
    name         = "boot"
    size         = "256M"
    start_sector = "8192"
    type         = "c"
  }
  image_partitions {
    filesystem   = "ext4"
    mountpoint   = "/"
    name         = "root"
    size         = "0"
    start_sector = "532480"
    type         = "83"
  }
  image_path                   = "raspberry-pi.img"
  image_size                   = "6G"
  image_type                   = "dos"
  qemu_binary_destination_path = "/usr/bin/qemu-arm-static"
  qemu_binary_source_path      = "/usr/bin/qemu-arm-static"
}

build {
  sources = ["source.arm.pi"]

  provisioner "ansible" {
    extra_arguments = [
      "--connection=chroot",
      "-e ansible_host=/tmp/rpi_chroot"
      ]
    playbook_file   = "setup-raspberry.yml"
  }
}

To run Packer builder ARM with Ansible in a one-liner use:

docker run --rm -it --privileged -v /dev:/dev -v ${PWD}:/build packer-builder-arm-ansible:vlatest build raspios.pkr.hcl

Or write a Docker Compose file named docker-compose.yml:

version: '3.9'

services:
  packer:
    image: packer-builder-arm-ansible:vlatest
    build: .
    privileged: true
    volumes:
      - /dev:/dev
      - .:/build
    command:
      - build
      - raspios.pkr.hcl 

Which is then simply run with:

docker-compose up

Once build process finishes successfully, we can test the result with what we used in part 1:

docker run --rm -it --privileged -v ${PWD}/raspberry-pi.img:/usr/rpi/rpi.img -w /usr/rpi ryankurte/docker-rpi-emu:latest ./run.sh rpi.img /bin/bash

Ansible outside of Packer

Once you have your provisioning playbook, it can be used on live Raspberry Pi devices as well. Command will be:

ansible-playbook setup-raspberry.yml --inventory <IPs>

Where IPs will be IP address of one or more devices you wish to modify:

• If more, use comma separator. For instance --inventory 192.168.1.162,192.168.1.163
• If only one, comma must be present at the end regardless. For instance --inventory 192.168.1.162,

If you are working with a growing number of devices, you will soon need to investigate about Ansible inventory and how to manage it. You might end up with something like this:

all:
  children:

    test:
      hosts:
        test1:
          ansible_host: 192.168.1.20
          hostname: test1-raspberrypi
        test2:
          ansible_host: 192.168.1.21
          hostname: test2-raspberrypi
      vars:
        purpose_type: test
        ntp_servers: ["europe.pool.ntp.org"]
        dns: [8.8.8.8, 8.8.4.4, 192.168.1.1]

    production:
      hosts:
        prod1:
          ansible_host: 10.3.0.25
          hostname: prod1-raspberrypi
        prod2:
          ansible_host: 10.3.0.26
          hostname: prod2-raspberrypi
      vars:
        ansible_ssh_common_args: '-o ProxyCommand="ssh -W %h:%p -q ubuntu@10.3.22.8"'
        ntp_servers: [10.3.0.1]
        dns: [10.3.0.1]

  vars:
    ansible_user: pi
    purpose_type: gpio_sensor
    screen_resolution: 1280x720ya

ansible_user and ansible_host define username and host for SSH connections to devices, which is used for management. Other SSH arguments can be defined as well.

But other variables don’t do anything by themselves — you need to use them in an Ansible playbook.

This was a short introduction to Ansible and what can you achieve with it. You can use it in place of a shell script, run Packer with it, manage a fleet of devices, encrypt secrets and so much more. When writing a playbook, you can always fallback to using a shell or command task.

Summary

In this article, several new concepts were introduced.

Firstly, we created a custom Docker image. That image extended one we used in part 1 and added Ansible installation.

We then defined an Ansible playbook. This playbook was used within Packer to create custom Raspberry Pi OS image. The playbook can also be used independently to modify live devices.

To run Ansible within Packer, HCL syntax was introduced for readability.

In the end, a new way of running a Packer build was introduced — Docker Compose. Compose is an excellent way of running Docker containers locally, especially for tools and testing.

This article ends 2 part series of packing custom Raspberry Pi OS with Packer and Ansible. Part 1 is available at https://medium.com/ascaliaio/part-1-custom-raspberry-pi-os-images-with-packer-testing-locally-18b4a03d44b.