K3s on OCI: A Kubernetes cluster in under 5 mins

K3s is a sandbox CNCF project that provides a lightweight Kubernetes that is optimized for edge and IoT use cases or for when you want to run Kubernetes but don’t want to spend that a lot of money, time or you don’t want to do Kubernetes — The Hard Way.

So let’s try this on OCI.

Single-server setup with an embedded DB

In this post, we want to see if it will work. The idea here is to make a simple deployment and poke around a little. Or if you need to run on the on the Oracle Cloud Free Tier in which case you can also try an K3s on ARM too.

For quick and dirty, we will need the following:

• a VCN
• an Internet gateway
• a public subnet and attached security list allowing SSH
• a route table for the public subnet to the Internet gateway
• a compute VM

So let’s use the terraform vcn module to do the dirtier work for us:

module "vcn" {
  source = "oracle-terraform-modules/vcn/oci"
  version = "3.0.0"
  # general oci parameters
  compartment_id = var.compartment_id
  label_prefix = var.label_prefix
  # gateways
  create_internet_gateway = true
  create_nat_gateway = false
  create_service_gateway = true
  # vcn
  vcn_cidrs = ["10.0.0.0/16"]
  vcn_dns_label = "k3s"
  vcn_name = "k3s"
  lockdown_default_seclist = false
}

Run terraform apply to create the VCN. Next, login to the OCI console and navigate to Networking > Virtual Cloud Networks and click on your VCN. Create a subnet, make sure you select the internet route table and the default options for DHCP and Security List. Give it a CIDR block too e.g. 10.0.0.0/24.

Once the subnet is created, navigate to Compute > Instances. Click Create Instance, selecting Oracle Linux 8, the k3s VCN and the public subnet you just created. Make sure the “Assign a public IPv4 address” is selected and upload/generate an ssh key. For the boot volume, let’s give it a generous 100GB (although we won’t be needing that much). If you are more inclined towards ARM, you can change the shape to Ampere and select VM.Standard.A1.Flex too.

Once the instance is available, ssh to the VM:

ssh -i /path/to/private/key opc@public_ip_address

Once logged in, we can install K3s:

curl -sfL https://get.k3s.io | sh -

Change the permission of the k3s.yaml file:

sudo chmod go+r /etc/rancher/k3s/k3s.yaml

Run kubectl:

kubectl get nodes

This should work.

Let’s install the Kubernetes dashboard:

GITHUB_URL=https://github.com/kubernetes/dashboard/releases
VERSION_KUBE_DASHBOARD=$(curl -w ‘%{url_effective}’ -I -L -s -S ${GITHUB_URL}/latest -o /dev/null | sed -e ‘s|.*/||’)
k3s kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/${VERSION_KUBE_DASHBOARD}/aio/deploy/recommended.yaml

Create the admin user and the cluster role binding and save to a file “dashboard.admin-user.yaml”:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

Create the user with admin rights and obtain the Bearer Token:

kubectl create -f dashboard.admin-user.yaml

Exit the ssh session and setup port forwarding:

ssh -L 8001:localhost:8001 opc@public_ip

Obtain the Bearer Token and run kubectl proxy:

k3s kubectl -n kubernetes-dashboard describe secret admin-user-token | grep ‘^token’
k3s kubectl proxy

Use your browser to access the dashboard at http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/

Ensure you select the Token option to login and paste the value of the Bearer token you obtained above.

And you can now login into your Kubernetes dashboard.

Conclusion

Well, this was hilariously easy: a Kubernetes cluster up and running in less than 10 mins while typing this up too. Admittedly, it’s a single node for everything so I wanted to see what I can achieve with a bit more automation: 4 mins and 11s. In a future post, we’ll experiment a bit more with other deployment options and use other OCI services.

References: https://rancher.com/docs/k3s/latest/en/installation/kube-dashboard/