Kubernetes on a Pi Cluster, Part 1
Getting familiar with Kubernetes is essential when working with modern software architecture as Kubernetes has evolved into the de-facto standard for container management and orchestration. Learning a technology in an environment where everything is working flawlessly together, will help you understand the basics. Every Kubernetes tutorial out there can teach you exactly that: “if everything goes well, Kubernetes will behave in a certain way”. But to truly understand a technology we have to go further! If things start crashing, we want to see how that technology handles those failures. And after we stopped crying, as our well designed system fails to recover, we can work on improving.
When using a hosted Kubernetes solution like Google Kubernetes Engine, it is hard to force nodes to fail or network-splits to occur (at least you will have a hard time convincing Google to force network outages or node failures, just to see whether or not the application can survive such a stituation). In addition, not everyone has a rack filled with servers in their basement, waiting to install Kubernetes and play around with.
This is exactly where the Raspberry Pi fits in. Having a desktop setup with Raspberry Pis, allows me to randomly turn off nodes, pull network cables and generally perform destructive actions — helping me to understand what happens with Kubernetes as well as my application in such situations.
In addition the Raspberry Pi is actually pretty affordable and in some cases ordering multiple might allow nice discounts. And as the current Pi (Rev. 4) is actually very powerful, it is a nice little server, feasible as a Kubernetes node.
These are pretty good sounding reasons to learn Kubernetes, right? But to be honest, I’ve yet another reason to get into Kubernetes on a Pi cluster: In some regards, I’m still a kid, that loves to play with toys. And the Raspberry Pi is an extremely interesting and versatile one. So let’s get started and play a bit!
Target Setup
In the end, the goal is to have all four Raspberry Pis as working nodes in a Kubernetes cluster. As a full fledged Kubernetes cluster is pretty complex, we’ll start with a k3s based cluster (https://k3s.io/). K3s is a lightweight Kubernetes distribution created by Rancher Labs, removing optional, alpha or legacy features (for details, have a look at the documentation of the GitHub repository https://github.com/rancher/k3s). In addition, k3s has support for ARM — a requirement when running on Raspberry Pi.
The intended setup will consist of a single Kubernetes master and three Kubernetes worker nodes. Every Raspberry Pi will be connected to the switch, which itself has an uplink connection to my normal network.
This article is the first in a series of articles. I’ll focus on the hardware setup of the Raspberry Pis in this part, leaving the Docker and Kubernetes installation for the upcoming parts.
The Hardware
As I wanted to be able to really mess with the cluster, like removing nodes, disconnecting the network and similar things, I chose a number of components to be part of the build (Beside those components the appropriate wires and AC plug):
- 4 Raspberry Pi 4 model B, 4GB RAM
- 1 Raspberry Pi Cluster Case
- 1 Netgear GS116E-200PES Switch
- 4 SanDisk microSDXC 128GB (160MB/s A2 C10 V30 UHS-I U3)
- appropriate wires and AC plug
Kubernetes requires the nodes to have at least 2 GB of RAM. Due to this, I’m using the 4GB model of the Raspberry Pi to have enough RAM available to run containers on that infrastructure.
Now with the hardware ready to run, it is time to prepare the SD cards with the appropriate image.
Flashing the SD cards
As the base image, I’m using Raspbian Buster. Those are meant as desktop images, but we will simply ignore the fact that X is running. The current version can be downloaded here: https://www.raspberrypi.org/downloads/raspbian/.
To write the image onto the SD cards, I’ve used the balenaEtcher (https://www.balena.io/etcher/). It is an amazing and straight forward tool and available for macOS, Linux and Windows. Nevertheless it was still a tedious job flashing all SD cards.
I want to play NOW!
I’m impatient. Really impatient. Of all the stuff I’ve ordered, not everything arrived at the same time. The switch took longer as well as the Micro HDMI adapters. Meaning, I could not connect all Raspberry Pis to my network (missing ports on my existing switch), neither could I inspect the running OS (missing Micro HDMI cables).
It turns out that the Raspbian team already thought of crying people in such a situation, and made it easy to enable the SSH daemon during first boot, as well as connecting to an existing Wifi network right away. To achieve this, two files have to be created and copied into the root of the SD card.
The first file is simply called ssh . This marker file will ensure that ssh will be available once the Raspberry Pi is started.
Second, I’ve created wpa_supplicant.conf , which contains the Wifi configuration like this:
In this file it is important to adjust country as well as the ssid and psk entries.
Interacting with the Raspberry Pis
Finally! All SD cards are prepared, inserted into the appropriate slots in the Raspberry Pis and they are ready to be started for the first time. As always, I expected something to go wrong (like typos in the configuration files, the SD cards to be unsupported, unsupported moon phase or generally anything to explode).
I was wrong! All Pis actually booted up, connected to the Wifi and were accessible using ssh. To log into Raspbian, use the username pi and password raspberry .
➜ ~ ssh -l pi 192.168.0.90
password:Linux raspberrypi 4.19.66-v7l+ #1253 SMP Thu Aug 15 12:02:08 BST 2019 armv7l[...]pi@raspberrypi:~ $
What’s next?
Now the Raspberry Pis are prepared and ready to be used as Kubernetes nodes. In the next part of this series, Kubernetes will be installed and started.
