Kubernetes Install Guide using Kubeadm on Oracle Cloud For Free.

Introduction.

This article is to a greater extent a procedural guide and restricted in clarifications on how things work, or why it is required — which the reference links ought to provide. Some fundamental information on SSH, cloud computing and DNS will be required.

Keep in mind that the kubernetes codebase is always evolving. As a result, this guide’s instructions and commands may soon become out of date and unusable. As of June 2023, the steps outline below ought to work successfully.

OCI free Service offerings.

Users may access a variety of cloud services with Oracle Cloud Always Free, a free tier offered from Oracle. Although it won’t always be free, customers may check out Oracle Cloud services, without paying anything as long as they stay within the bounds that Oracle sets. Free services are; Compute Instances, Security, Networking, Database, Storage.

Free Compute instances on OCI:
2x AMD Instances with 1GB each
Up to 4x ARM instances with 4 cores and 24GB split between them

For my Kubernetes cluster:

1x 2 OCPU ARM 12GB — control plane
1x 2 OCPU ARM 12GB — worker node

prerequisite:

1. Create an OCI account:
   Create an account by going to https://www.oracle.com/cloud/. The process is fairly straightforward.
2. Create VMs:
   For the 2 nodes, I used Ubuntu Jammy 22.04 (LTS). 2 ARM instances, each with 2 OCPUs and 12 GB, were created. For more information on how to create a compute instance in OCI, please visit oracle-base.

Control plane and worker nodes.

3. Local DNS:
Update the /etc/hosts/ file of both nodes. Add a line with the node IP address, FQDN and hostname. e.g.

#These entries should be in both server's "hosts" file.
10.245.x.x <master-node-fqdn> <master-node-shortname>
10.245.x.x <worker-node-fqdn> <worker-node-shortname>

4. Firewall:
OCI has some predefined rules in ubuntu instances iptables. These rules can block some ports needed for Kubernetes setup and networking. Apply these commands on each of the nodes.

#save existing rules
sudo iptables-save > ~/iptables-rules

#modify rules, remove drop and reject lines
grep -v "DROP" iptables-rules > tmpfile && mv tmpfile iptables-rules-mod
grep -v "REJECT" iptables-rules-mod > tmpfile && mv tmpfile iptables-rules-mod

# apply the changes
sudo iptables-restore < ~/iptables-rules-mod

# confirm that there are no rules
sudo iptables -L

#save the changes
sudo netfilter-persistent save
sudo systemctl restart iptables

Setting up kubernetes using KubeAdm.

1. Install a container runtime. We will use Docker engine.

#Update the apt package index and install packages to allow apt to use a repository over HTTPS:
 sudo apt-get update
 sudo apt-get install ca-certificates curl gnupg

#Add Docker’s official GPG key:
 sudo install -m 0755 -d /etc/apt/keyrings
 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
 sudo chmod a+r /etc/apt/keyrings/docker.gpg

#Use the following command to set up the repository:
 echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

#Note: If you use an Ubuntu derivative distro, such as Linux Mint, you may need to use UBUNTU_CODENAME instead of VERSION_CODENAME.

#Install Docker Engine
#Update the apt package index:
 sudo apt-get update

#Install Docker Engine, containerd, and Docker Compose.
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

#Verify that the Docker Engine installation is successful by running the hello-world image.
sudo docker run hello-world

2. Install cri-dockerd. This adapter provides a shim for Docker Engine that lets you control Docker via the Kubernetes Container Runtime Interface.

#Install GO for compling. Because we are using an ARM architecture, we need to download the ARM installation file for Linux.
wget https://go.dev/dl/go1.20.5.linux-arm64.tar.gz

#Now install GO using the command:
sudo tar -C /usr/local -xzf go1.20.5.linux-amd64.tar.gz

#Add /usr/local/go/bin to the PATH environment variable.
#You can do this by adding the following line to your $HOME/.profile or /etc/profile (for a system-wide installation):

export PATH=$PATH:/usr/local/go/bin

#Verify that you've installed Go by opening a command prompt and typing the following command:
go version


########### Installing cri-dockerd ########################

#To begin following the build process for this code, clone this repository in your local environment:
git clone https://github.com/Mirantis/cri-dockerd.git

#To build this code
cd cri-dockerd
make cri-dockerd

#To install, on a Linux system that uses systemd, and already has Docker Engine installed

# Run these commands

mkdir -p /usr/local/bin
install -o root -g root -m 0755 sudo ./cri-dockerd /usr/local/bin/cri-dockerd
install packaging/systemd/* /etc/systemd/system
sudo sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service
sudo systemctl daemon-reload
sudo systemctl enable cri-docker.service
sudo systemctl enable --now cri-docker.socket

3. Installing kubeadm, kubelet and kubectl.

#Update the apt package index and install packages needed to use the Kubernetes apt repository:

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

#Download the Google Cloud public signing key:
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg

#Add the Kubernetes apt repository:
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

#Update apt package index, install kubelet, kubeadm and kubectl, and pin their version:
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

#Note: In releases older than Debian 12 and Ubuntu 22.04, /etc/apt/keyrings does not exist by default. You can create this directory if you need to, making it world-readable but writeable only by admins.

4. Initializing the control plane node using kubeadm. We are going need a pod network using calico to initialize the control plane node.

#To initialize the control-plane node run:
sudo kubeadm init --pod-network-cidr=192.168.0.0/16 --cri-socket=unix:///var/run/cri-dockerd.sock

#We need to specify a container runtime domain sock as well. 
#Since we are using docker enginer, we specify the "--cri-socket" in the command above.

5. A successful kubeadm initialization will have the following message at the end. Ensure you note the details for joining the worker node to the cluster.

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join <ip address>:6443 --token <value>\
        --discovery-token-ca-cert-hash sha256:<value>

6. To complete the installation of your pod network using calico. Use the commands below.

#Ensure you're in the home directory
cd ~

#Download the calico pod network yaml file
curl https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml -O

#Install on the master node using.
kubectl apply -f calico.yaml

#To check all pod status, run:
kubectl get pods -A

7. Repeat the periquisite steps 1 to 4 and setting up kubernetes using KubeAdm steps 1 to 6 on the worker node(s).

8. Finally, join the worker node to the control plane node using the commands below.

#remove the kubernetes certificates created on the worker node.
rm -rf /etc/kubernetes/pki/

#Remove the kublet config file.
rm -rf /etc/kubernetes/kubelet.conf

#Reset the node.
Kubeadm reset

#Join the worker node to the control plane node.
kubeadm join --token <token> <control-plane-host>:<control-plane-port> --discovery-token-ca-cert-hash sha256:<hash>

#Confirm the worker node is joined to the cluster by running the command below
#on the control plane node.
kubectl get nodes

#To label the worker node. Run:
kubectl label nodes slave-node kubernetes.io/role=worker

Conclusion.

The two VMs used for the cluster are ARM64 architecture based and any binary installed or container image from Docker installed should be ARM64 based.