Transform Your Raspberry Pi: Lightweight Power with k3s and Cilium!

¡Hi! I'm thrilled to share this tutorial with you. It was born from my personal journey of installing a unique setup on my Raspberry Pi. I chose k3s and Cilium for their incredible lightness and efficiency. This setup has transformed my Raspberry Pi into a more powerful and versatile device, perfect for experimenting and learning. I'm eager to walk you through every step, showcasing how these tools can enhance your Raspberry Pi experience. Your contributions are not just welcome, but highly appreciated, as they enrich this learning journey for all of us. Let's dive into this exciting world together and unlock new possibilities with our Raspberry Pis! 🚀🤓

Configuring Raspberry

Get the router IP and define your Raspberry Pi host CIDR.

## this will return the router ip
ip route show | grep -i 'default via'| awk '{print $3 }'

Setup our local DHCPD to have a static IP address by modifying the button at /etc/dhcpcd.conf. ROUTER_IP and IP_ADDRESS

slaac private
interface eth0
static routers=<ROUTER_IP>
static domain_name_servers=200.21.200.10
static ip_address=<IP_ADDRESS>

We are going to install K3s, following the requirements for the Raspberry Pi. Standard Raspberry Pi OS installations do not start with cgroups enabled. K3S needs cgroups to start the systemd service. cgroups can be enabled by appending cgroup_memory=1 cgroup_enable=memory to /boot/cmdline.txt.

cat << EOF > /boot/cmdline.txt
console=serial0,115200 console=tty1 root=PARTUUID=58b06195-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_memory=1 cgroup_enable=memory
EOF

Reboot the system

sudo reboot

Installing K3S

Then, we need to install K3S cli with a server and agent. We are going to disable multiple network functions because we are going to use Cilium; metrics-server will be installed later.

curl -sfL <https://get.k3s.io> | sh -s - --write-kubeconfig-mode 644 --flannel-backend=none --disable servicelb --token some_random_password --disable-network-policy --disable "metrics-server" --bind-address <IP_ADDRESS> --disable-cloud-controller --disable local-storage --disable "traefik"

sudo cat /etc/rancher/k3s/k3s.yaml > ~/.kube/config

Check the master node until it gets ready.

kubectl get nodes

Cluster is Ready!

Install cert-manager

We are going to use certmanager to create TLS certificates for ingress and internal communication.

Create the configuration file:

cat << EOF > cert-manager-config.yaml
resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 100m
    memory: 128Mi
installCRDs: true
webhook:
  hostNetwork: true
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 100m
      memory: 128Mi
cainjector:
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 100m
      memory: 128Mi
EOF

Install with helm

helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install \\
  cert-manager jetstack/cert-manager \\
  --namespace cert-manager \\
  --create-namespace \\
  --version v1.13.2 \\
  --values cert-manager-config.yaml

Disable cert-manager validation (assuming Cilium will be installed in the kube-system namespace):

kubectl label namespace kube-system cert-manager.io/disable-validation=true

Create a cluster issuer for Hubble.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: ca-issuer
spec:
  selfSigned: {}

Install Cilium

CLI installation

CILIUM_CLI_VERSION=$(curl -s <https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt>)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all <https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}>
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

Install Cilium in cluster

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

cilium install --version 1.14.4

Check Cilium status

cilium status

Enable Hubble

cilium hubble enable

cilium status

Install Hubble cli

HUBBLE_VERSION=$(curl -s <https://raw.githubusercontent.com/cilium/hubble/master/stable.txt>)
HUBBLE_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; fi
curl -L --fail --remote-name-all <https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}>
sha256sum --check hubble-linux-${HUBBLE_ARCH}.tar.gz.sha256sum
sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin
rm hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

cilium hubble port-forward&
hubble status

It should return:

Configure Hubble with TLS and cert-manager

## file /root/.config/hubble/config.yaml
mkdir /root/.config/hubble
cat << EOF > /root/.config/hubble/config.yaml
basic-auth-password: ""
debug: false
server: localhost:4245
tls:
 auto: enabled
 certmanagerissuerref:
   group: cert-manager.io
   kind: ClusterIssuer
   name: ca-issuer
 certvalidityduration: 1095
 method: certmanager
tls-ca-cert-files: []
tls-client-cert-file: ""
EOF

Create and replace with IP_ADDRESS from the first step CiliumLoadBalancerIPPool

cat << EOF > cilium-lb-pool.yaml
apiVersion: "cilium.io/v2alpha1"
kind: CiliumLoadBalancerIPPool
metadata:
  name: "lb-raspberry"
spec:
  cidrs:
  - cidr: "<IP_ADDRESS>/30"
EOF
kubectl apply -f cilium-lb-pool.yaml

enable Hubble UI

cilium hubble enable --ui

Expose Hubble UI with Load Balancer

cat << EOF > hubble-lb.yaml
apiVersion: v1
kind: Service
metadata:
  name: hubble-lb
  namespace: kube-system
spec:
  loadBalancerIP: <IP_ADDRESS>
  type: LoadBalancer
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8081
  selector:
    k8s-app: hubble-ui
EOF
kubectl apply -f hubble-lb.yaml

Now you can access to Hubble!

As we wrap up this journey, I want to extend a heartfelt thank you for joining me in exploring the transformative power of k3s and Cilium on the Raspberry Pi. Your engagement and contributions have made this not just a tutorial, but a vibrant community learning experience. Stay curious, keep experimenting, and remember, the world of Raspberry Pi offers endless possibilities. Until next time, happy tinkering! 🌟👩‍💻🚀

Thank you for reading, and I hope this has been helpful to you. See you soon in this ongoing saga!

https://www.linkedin.com/in/juanfbl9307/

https://tritontechsolutions.com/

https://github.com/juanfbl9307