Kubernetes The Easy Way

Published in

devopsanswers

9 min readAug 21, 2022

Kubernetes is the go-to solution for microservices and container-based production implementations, which is trusted by the community and large web-scale companies. It’s backed and used by titans like Red Hat, IBM and Microsoft.

When I first got to know Kubernetes, it was not a very simple concept to understand as we were working on bare-metal and hypervisor-based production implementations every day.

Industry experts like Kelsey Hightower predict Kubernetes will be the next big thing after the hypervisors and the cloud era. So I thought of simplifying the Kubernetes concepts in easy to understand manner to help others who are eager to learn. I will explain the main components and practical usage by using exciting visuals wherever possible.

Let’s begin the very first article of my Kubernetes series, Kubernetes - The Easy Way.

Pro Tip!

Please come out of your Comfort Zone and be in your Learning Zone to Ace the Kubernetes if you don’t have any basic understanding or experience before. There might be a lot of questions, but don’t give up :)

A little bit of History

Kubernetes was born as a result of Google’s decade of experience in managing containerized systems at a large scale using Borg and Omega. They used to run hundreds of thousands of jobs, from many thousands of different applications, across many clusters. Later, Google introduced Kubernetes as an open-source version of Borg.

Kubernetes and Containers

Containers (Docker, rkt, Containerd) allow us to create, deploy, and run applications in a very effective way. It will package an application with required libraries and other dependencies, and ship it all out as one package. Not like Virtual Machines, it will share the same host operating system kernel across the containers.

In production, we need to ensure all the services will run with maximum availability. If a container dies, another container should spawn and continue the workload. But how to achieve this state will be a question if you are new to the container-centric world. A container orchestration system which will provision, schedule and manages containers at scalable manner would be the ideal choice.

That’s where Kubernetes will be the trusted platform to orchestrate, scale and failover the containerized applications. It adds more features like automating app deployments and updates, health-check and self-heal apps with autorestart, autoreplication and autoscaling.

The Easy Way

Kubernetes Main Components as an example of a Sea Port — “Seaport Infographic” extracted from freepik.com by macrovector

Kubernetes contain a few main components as seen from the simplified diagram above, which displays a typical operation of a Maritime Port facility. I wanted to make things much easy to understand for everyone. So I will explain the main concepts using the above diagram as the starting point.

You could see that the Ships are doing the hard work of moving containers across the sea. The Main Control Center is responsible for communication, managing containers and monitoring the Ships.

According to the Kubernetes analogy, Ships will consider as Worker Nodes which can load containers.

The Main Control Center will load the containers to the Ships and identify which containers should go into which Ships. In addition, it will plan how to load containers, store information about the containers and ships, monitor the containers and ships including the loading and unloading process.

The Main Control Center has different departments to handle various tasks such as loading and moving containers between ships, monitoring the containers and workload, tools such as cranes to move containers and devices to communicate between ships and Main Control Center.

The Main Control Center will consider as the Kubernetes Master according to the Kubernetes terminology.

I will walk through each component briefly.

Master

Kubernetes Master (control plane) is the main component of managing a Kubernetes cluster deployment. According to the diagram, the Main Control Center of the port will be considered as our Master node.

Master has a few more components such as ETCD cluster, Kube API Server, Kube Controller Manager and Kube Scheduler which will combine together to control the Kubernetes cluster.

We need at least three masters to run a production Kubernetes cluster to facilitate high availability.

ETCD Cluster

There is much information you need to store regarding the port’s daily operations. The number of ships comes to the port, the number of containers loaded and unloaded, container load and unload timestamps and which ships handled which containers? etc. We need to ensure this data is recorded somewhere and available on-demand. The Data Store Facility in the Port will store all of the data.

In Kubernetes terms, the Data Store is considered as the ETCD cluster. It’s basically a key-value based distributed data store. It will actually store the critical data related to the Kubernetes cluster such as config data, cluster state and metadata.

Kubernetes use ETCD functionalities to monitor the cluster changes. When you interact with the Kubernetes cluster using the API, you will read the command output values (kubectl get) which are stored in ETCD. Same way, when you use API to create Kubernetes resources (kubectl create), it will write back to ETCD. So if you want to back-up cluster data, ETCD is the right pick.

Pro Tip!
To backup a Kubernetes Cluster you can use a tool like Velero from VMware or k8up which is an opensource project by Devops wizards at VSHN

Kube API Server

The API server will act as the backbone of the communication channel between the Kubernetes components. The end user can use kubectl cli tool to manage the cluster deployments via API calls. When you send a command via API endpoint, k8s cluster will set this as the desired state of new or existing components.

The Communication Tower in port will be considered as the API Server which facilitates the communication between the ships and the control center.

Kube Controller Manager

This component manages various controllers in Kubernetes. Controllers are control loops that continuously watch the state of your cluster, and then make or request changes where needed. Each controller tries to move the current cluster state closer to the desired state.

In our example, Port Controller Room will be considered as Controller Manager.

There are number of controllers available. I’ll list down a few controllers with the usage.

Node Controller - responsible for monitoring the status of the nodes and taking necessary actions
Replication Controller - monitoring the status of replica sets and ensuring that the desired number of PODs are available at all times within the ReplicaSet
Deployment Controller - monitoring the status of the deployment and ensuring that the desired number of PODs are available at all times
Job Controller - responsible for monitoring the status of the jobs and updating the job status to API Server

Kube-Scheduler

The kube-scheduler is responsible for assigning pods to nodes in the cluster by making the decision of which pods go to which nodes.

In our example, the container handling crane is making sure which container goes to which ship.

Scheduler is only the decision maker for the pod assignment. Actual heavy lifting is done by kubelet by running the pods as underline containers.

The scheduler will go through each pod to identify CPU and Memory requirements and it will skip the nodes which don’t have enough resources to run the containers.

The scheduler follows a ranking method for the remaining nodes to finalize a node to fit the pods. It prioritizes the node by following a scale which calculates the remaining resources, if it scheduled the pods in that node. The node which is having the highest number of free resources wins and gets a better rank.

The scheduler can be used to control the pod placement of the nodes using nodeSelector and Node affinity rules.

Worker Nodes

A worker node is basically a compute instance that will run the cluster workload as containers. There could be thousands of worker nodes in a high-end Kubernetes cluster.

According to the example, Ships will be the worker nodes.

It has important components such as Kubelet, Kube Proxy and Container Runtime Interface (CRI).

Kubelet

Kubelet is the main component of the worker node, which manages the pod lifecycle.

In our example, the ship captain will be considered as the Kubelet. He will continuously communicate with the Main Control Center to give information about the remaining cargo space in the ship, destination country, fuel requirements etc.

In return, the Control Center will send instructions to the Crane operator (Scheduler) to stack containers which fit the ship's remaining space and destination.

In an actual k8s cluster, Kubelet communicates with the Master node via API Server and gives information about the remaining CPU, Disk and Memory resources. Also, Kubelet is the component which joins the node to the Kubernetes cluster by talking to the Master API endpoint and initiating the TLS handshake.

Master will send the instructions to Kubelet via API Server to run the POD. Kubelet will run the underline containers using the container run time engine (Docker/Containerd) by pulling the required Docker images.

The Kubelet then continues to monitor the state of the POD and reports to the Master via Kube API.

Refer to the highlighted parameters in the following Kubelet config file to understand how the Kubelet configures the client CA certificate, sets cluster DNS (for PODs) and Kubernetes manifest location to contain the static pod definitions(etcd, kube-apiserver, kube-controller-manager and kube-scheduler).

apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 0s
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
cgroupDriver: systemd
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
kind: KubeletConfiguration
resolvConf: /run/systemd/resolve/resolv.conf
rotateCertificates: true
staticPodPath: /etc/kubernetes/manifests

Kube-Proxy

There is an interesting thinking behind the introduction of kube-proxy. Let’s have a look at the practical approach of the kube-proxy and what is the exact usage of this component.

Kubernetes Pods are designed to terminate and relaunch according to the Node status to facilitate self-healing and high availability of the running application. So if we use a k8s Deployment or ReplicaSet, it will create and destroy the pods dynamically to maintain the desired state.

So if k8s rely on pod IPs to connect with other pods, it’s not a practical solution as the IPs keep changing on each new pod. So if you want to reach another pod without using its IP, you need to expose the pod as a Service.

In a k8s cluster each pod should be able to reach all other pods by default. This can be controlled using Network Policies in a production cluster to properly isolate pods.

Here comes the fun part; if we keep creating new services, there should be a way to keep track of each service from each node. So two pods can communicate with each other regardless of the current node the pods are running.

The solution is Kube-Proxy. It looks for new services and it creates the appropriate rules on each node to forward traffic to those services to the backend pods. In a legacy cluster, kube-proxy will use Linux iptables rules to forward traffic.

Kube-proxy is deployed as a process that runs on each node in the Kubernetes cluster as a DaemonSet.