Setup a Kubernetes K3s Homelab with Armbian Ubuntu

Pine64 as a K3s primary node

So you’re ready to take the plunge? What are you waiting for? Let’s do this!

I’m on a journey to move all of my home services to a homelab. I’ve created this guide and Github repo to capture the steps I used to setup a K3s homelab. Hopefully I’ll save you time on yours.

Here are the things you will need before you can get started:

• At least two arm based devices (Pine64, Raspberry PI, etc…)
• MicroSD Flash Drives
• Network (cables, network switch)
• A router somewhere in your network which you have admin access to
• A device to work from (Macbook Pro or ‘nix based recommended)

You’ll also need to give consideration to your network. Depending on number of nodes, you’ll need a properly sized network switch and/or you may prefer to setup a dedicated router for your cluster. In my use case a dedicated router was overkill, so I am using my primary network router.

Each node will need its own statically assigned IP. For simplicity of getting started, I recommend using DHCP and setting each device to static lease type as it comes online. This uses the device MAC address for static IP and hostname assignment.

Now, let’s get started.

First you need to get your MicroSD cards ready. The image you will need depends on the device. I’m using Armbian community images and the rest of this guide will assume you are using these. Search the community images for your needs. I need Pine64 and SOPine.

• Pine64 Kinetic Ubuntu Minimal Community Image
• SOPine Kinetic Ubuntu Minimal Community Image

Now, either use the CLI (below) OR script OR download to your workstation and use a UI based option like Etcher.

If using CLI, follow the steps below to become root and change the variables to match your needs.

sudo su -

export DOWNLOAD_URL="https://github.com/armbian/community/releases/download/202249/Armbian_23.02.0-trunk_Pine64_kinetic_edge_6.0.10_minimal.img.xz"
export ARCHIVER="xz -d"
export DEVICE=/dev/disk2

diskutil unmountDisk $DEVICE
curl -L -f "$DOWNLOAD_URL" | $ARCHIVER | dd of=$DEVICE

Tip: If you’re flashing several different device types, mark your cards so you don’t mix them up!

Load the MicroSD card whichever device will be your Kubernetes primary node and boot it up. If you’re using the Armbian minimal community images like suggested, connect to it as root@<machine-ip>, enter 1234for the password when prompted. You will then be prompted to create a new user, password, and a few other things.

First things first we need to set the hostname. Pick a name that tells you this is the primary (aka master).

sudo hostnamectl set-hostname homelab-primary
sudo perl -i -p -e "s/pine64/homelab-primary/g" /etc/hosts

Go find this host in your router, typically in IPv4 addresses or DHCP assignment tables and set it to “static lease”. Also, you may need to change the hostname here in your router to match the new hostname you set.

Kubernetes runs docker images, so we had better install docker:

sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker

Great, now we can install Kubernetes. In this guide we’ll use K3s which is a lightweight 100% compatible binary of Kubernetes. Install as root!

sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.25.4+k3s1 sh -

Let’s check if we’re up

sudo kubectl get nodes

You should see output like this:

Fetch the K3s node token and save it for later (you’ll need it on the worker):

cat /var/lib/rancher/k3s/server/node-token

Now we need to setup our work node(s). Boot it up and go through the initial user/password prompts.

Set the hostname to something logical like we did on the primary

sudo hostnamectl set-hostname homelab-sopine-01
sudo perl -i -p -e "s/sopine/homelab-sopine-01/g" /etc/hosts

Go find this host in your router, typically in IPv4 addresses or DHCP assignment tables and set it to “static lease”. Change the hostname if needed.

Install docker:

sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker

Open a new terminal and connect back to your primary to fetch the node token:

cat /var/lib/rancher/k3s/server/node-token

Also, you’ll need the k3s.yaml from your primary and to place it into ~.kube/config on each node.

cat /etc/rancher/k3s/k3s.yaml 

~.kube/config

Now lets install a matching version of K3s on the worker node, replace <K3S-NODE-TOKEN> with the K3s node token you saved earlier. Note that for this to work your primary node must be resolvable from your worker and must have its firewall down, or required ports like 6443 open.

curl -sfL https://get.k3s.io | \
 K3S_URL=https://homelab-primary:6443/ \
 K3S_TOKEN=<K3S-NODE-TOKEN> \
 INSTALL_K3S_VERSION=v1.25.4+k3s1 sh -

Return to your terminal on your primary and check both primary and your node:

homelab-primary:~:# kubectl get nodes
NAME                 STATUS   ROLES                  AGE     VERSION
homelab-primary      Ready    control-plane,master   1h   v1.25.4+k3s1
homelab-sopine-01    Ready    <none>                 2m   v1.25.4+k3s1

Hopefully you’re seeing your node up and Ready! It may take a few minutes, so if you see “Not Ready”, give it a few minutes and check again. If you’re running into any trouble at this point I recommend digging more into the K3s docs, starting with requirements and quick-start.

I’ve made a homelab repo with several setup scripts that speed up the setup process. Read the Readme for details.

From here I’ll leave you with a few resources that helped me start using the cluster.

Kubectl cheatsheet is a must!

Kubernetes-dashboard, use the helm chart.

If you need helm, it’s easy to setup. Install kubectl and helm on your local, make sure kubectl is a close version match to your nodes (1.25.4). Oh and make sure you get your k3s.yaml into ~.kube/config on your local.

Next time I’ll cover ArgoCD and I’ll use it to deploy kubernetes-dashboard.