Comprehensive Guide to Container Orchestration (Kubernetes )

Dar
16 min readJan 10, 2024

--

Kubernetes is an open-source container orchestration platform designed to automate the deployment, scaling, and management of containerized applications.

K8s Architecture:

Master Node:
It runs several K8 processes that are necessary to run & manage the cluster. In production environment, it is mandatory to have atleast 2 Master nodes at any time in a K8 cluster.
Components:
- API Server: A container, which is entry point to K8 cluster. To manage the cluster, we can talk to the API Server using UI, CLI or API(Automating Scripts) clients.
- Controller Manager: It keeps tracks of everything happening in the cluster. i.e. Container Repair, Restart etc.
- Schedular: An intelligent process responsible for scheduling containers on different nods based on the available server resources. i.e. Ensures Pods placement.
- ETCD: A key value storage component, which holds the current state of K8 cluster at any time. It holds config and status data of each node and each container inside of that node. i.e. In the event of a disaster, we can recover the whole cluster state using the ETCD snapshot.

Worker Nodes:
Docker containers with different applications are deployed on worker nodes. Each worker node can have 100’s of containers.
Components:
- Kubelet: A K8 process, that makes it possible for cluster to communicate with each other and execute tasks.
- Pods: Pod is the smallest unit that we can configure & interact with. We don’t directly create/configure containers in K8 cluster. Each Pod is it’s own self contained server(ip addr). We only work with Pods, which is an abstraction layer over containers and it manages the containers running inside it without our intervention. i.e. when a container dies inside a Pod, it will automatically be restarted.
- Service: Instead of having dynamic IP addresses for Pods, Services sits directly Infront of each pod, that talk to each other. When a Pod dies, it is replaced by a new pod but the IP address stays the same because it is managed by service component. Service component has two functionalities (Permanent IP address, Load Balancing).

Virtual Network:
Master & work nodes talk to each other through a virtual network.

K8 Cluster Setup with Minikube:
Minikube provisions and manages Kubernetes clusters optimized for development workflows on local workstations. It creates a virtual machine on the local system and deploys a Kubernetes cluster within that VM. It provides a local environment for testing and developing Kubernetes applications without the need for a full-scale production cluster.

# Dependency-> VM VBox or Hyper-V
cd /d/devops
mkdir kubernetes
cd kubernetes

# Cloning project from Github
# git clone https://github.com/zzd786/vprofile-project.git
# cd vprofile-setup
# git pull
# git checkout kubernetes-setup
# cd minikube

#{Open Power Shell as Admin
# Chocolatey Setup
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
# MiniKube Setup
choco install minikube kubernetes-cli -y #}

minikube.exe --help

# Start the K8 Cluster. It will download an image & start a VM.
minikube start

# If you get an error related to VT-X/AMD-v, Open VM Box in background & try
minikube start --no-vtx-check

# To check nodes (Control-plane is the Master node)
kubectl get nodes

# Config file in Mini Kube
~ cat .kube/config
# OR
kube config view

K8 Cluster Setup with KOPS:
Kops is a tool used to provision and manage K8's clusters on various cloud platforms(AWS, GCP). Cluster configuration is defined in a YAML file & then Kops uses this configuration to create and manage the necessary cloud resources and Kubernetes components.

# Dependencies: 
# A Domain for DNS records
# Create a Linux VM to launch the K8 Cluster with Kops (EC2 Instance)
# AWS Account (S3 Bucket, IAM user for AWS CLI, Route 53 Hosted Zone)

1) Creating an EC2 Instance for Kops:
Name: kops
AMI: Ubuntu 22.04
Instance type: T2.micro
Key Pair: kops-key
Network settings: Ip->My IP, Security group-> kops-sg, Description-> kops-sg

2) Creating an S3 bucket:
Name: kops-bucket-786
Region: Same as EC2 instance

# Kops will create the cluster & use AWS Cli behind the scene to access the AWS
# services. In order to run the commands, I need to give credentials. For that I
# need an IAM user.

3) Creating an IAM user:
Name: kops-admin

# I can attach individual polies or give admin access according to need.
# I will give admin access since kops-admin user will be managing EC2, S3 & Route 53 services.
Attach-Policies-Directly: Administratoraccess

# To generate a secure key for CLI Access. Click;
kops-admin: Security Credentials ->Create Access Key-> CLI -> Save.csv file for later.

4) Creating Route 53:
- Create Hosted Zone
- Create a subdomain name
- kube.example.online
- Add AWS NS Values into the DNS records on Godaddy as a NS.
- Type: NS, Name: kube, Value: AWS NS URL

# Use www.whatsmydns.net to validate the domain name.
# OR
nslookup -type=ns kube.example.online

Login into EC2 Instance using GitBash and configure AWS CLI. After that, setup Kubectl & Kops.

cd /d/devops/aws-kops
# Copy Public IPV4 address of the EC2 instance from AWS
ssh -i kops-key.pem ubuntu@1.2.3.4

# Generating an SSH Key which will be used by Kops
ssh-keygen

# Install AWS CLI
sudo apt update && sudo apt install awscli

# Setting up AWS ClI on my local Machine (Ubuntu->EC2). I can get access to
# AWS API by entering AWS secret key of kops-admin IAM user.
# If I don't do this, I can still navigate file system, run applications(locally)etc.
# But I won't be able to access or manage any AWS resources from the
# EC2 instance, including instances, S3 buckets, RDS database.
aws configure
# To check
aws configure list
# Setup Kubectl: This will download the binary.
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

# Make the binar executable
chmod +x ./kubectl
ls -l

# Move it to /usr directory to access it from anywhere. This directory has
# permissions like rwxr-xr-x (or 755), allowing any user to read and execute files.
sudo mv kubectl /usr/local/bin

# Installing Kops. Latest releases have bugs so I will install v1.26.4.
wget https://github.com/kubernetes/kops/releases/download/v1.26.4/kops-linux-arm64

# Make the binary executable and move it and rename to kops.
# Renaming it to kops makes it easier to reference kops command.
chmod +x ./kops-linux-amd64
sudo mv kops-linux-amd64 /usr/local/bin/kops

Now, I will create the Kubernetes cluster using Kops command on AWS.

# To check the available zones in a region
aws ec2 describe-availability-zones --region eu-central-1

# The command will create configuration for the cluster & store it in S3 bucket.

# --name: cluster name that is a valid fully-qualified DNS name.
# --state: S3 bucket where kops will maintains the configuration & state information of the cluster
# --zones: The availability zones for the cluster nodes.
# --node-count: The number of worker nodes in the cluster.
# --node-size: The instance type for worker nodes (vCPU, RAM etc).
# --master-size: The instance type for master nodes.
# --dns-zone: The DNS zone for the cluster.
# Size must be specfied, otherwise AWS will by default allot 120gb which will result in big cost.
# --node-volume-size: The size of the volume(internal) for worker nodes.
# --master-volume-size: The size of the volume(internal) for master nodes.

kops create cluster --name=kube.example.online \
--state=s3://kops-bucket-786 --zones=eu-central-1a,eu-central-1b \
--node-count=1 --node-size=t3.small --master-size=t3.small \
--dns-zone=kube.example.online --node-volume-size=8 --master-volume-size=8

# To update and launch the cluster
# --admin: Grant administrative privileges to the IAM user whose credentials are configured with the aws configure command.
# The admin will have full access to the cluster's control panle nodes.
kops update cluster --name=kube.example.online --state=s3://kops-bucket-786 --yes --admin

# To get a list of configured clusters from S3
kops get cluster --name=kube.example.online --state=s3://kops-bucket-786

# To validate the cluster (Overall health)
kops validate cluster --state=s3://kops-bucket-786
kubectl get nodes

# To Delete the cluster
kops delete cluster --name=kube.example.online --state=s3://kops-bucket-786 --yes

# To shutdown Kops VM or EC2 instance
sudo poweroff

Kubeconfig File:
When We create K8 cluster using Kops or some other method. We get a file called Kubeconfig. It contains info about 4 things;
- Users
- Clusters
- Namespaces
- Authentication
Kubectl needs all the information to work. This info is stored in .kube/config file in YAML format. Kubectl uses the server URL to talk to the API server in the Master node.

cd ~
ls -a
cat .kube/config

# Context binds the cluster with the Users in the config file.
# To check KubeCtl context
kubectl config view

# To run kubectl from windows.
# Install Kubectl on windows using chocolatey
cd ~
mkdir .kube
vim .kube/config
# Copy the config content from ubuntu running in EC2 to the config on Windows.
# And start using kubectl on windows.
# I can use the kubeconfig file anywhere, i.e. I can use it in Jenkins to
# apply or deploy changes to K8 objects(PODS, Service, Deployment etc).

Namespaces:
To group or isolate the resources, we can use namespaces. I can create dev namespace for development environment & prod namespace for my production environment to isolate the resources within a cluster. Names should be unique in a single namespace.

# To check existing namespaces
kubectl get ns

# To get all the services from all the namesapaces
kubectl get all -all-namespaces

# To get services or pod form a specific namespace
kubectl get svc -n kube-system
kubectl get pod -n kube-system

# To create my own namespace & a pod
kubectl create ns zzdnamespace
kubectl run nginxpod --image=nginx -n zzdnamespace

# We can also do this my using a definition file
vim pod1.yaml
# Add
apiVersion: v1
kind: Pod
metadata:
name: nginxpod
namespace: zzdnamespace
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80

# To apply changes
kubectl apply -f pod1.yaml
kubectl get pod -n zzdnamespace

# To delete a namespace (Dangerous: Everything related to this ns will be gone)
kubectl delete ns zzdnamespace

PODS:
The smallest unit in K8 cluster. A Pod represents processes running on our K8 cluster.
- Single Container POD: Most common model. Pod works as a wrapper around a container and K8 manages the pods rather then the container directly.
- Multi container POD: It has 1 main container & other helper containers. It is tightly coupled and resources inside a pod are shared.
We can create a separate pod for each service; Tomcat, Nginx, MySQL, RabbitMQ etc. For high availability, we can create multiple pods for a single service i.e. Tomcat pods and manage them with K8.

In K8 cluster, we can use a DEFINITION file to create multiple PODS in one go just like we used DOCKER COMPOSE to create multiple containers.

mkdir definitions
cd definitions
mkdir pods
cd pods

# I am creating a Pod named "vpro" with a single container named "d-nginx"
# using the Docker image "zzd786/d-nginx"
# The container exposes port 8080, and the Pod is labeled with "app: vproapp"
vim vpro.yaml
---
apiVersion: v1
kind: Pod
metadata:
name: vproapp
labels:
app: webserver
spec:
containers:
- name: tomcat
image: zzd786/d-application
ports:
- name: tomcat-port
containerPort: 8080
# Creating POD. -f is to describe the filename containing K8 definitions
kubectl create -f vpro-pod.yaml
kubectl get pods
kubectl describe pod vproapp

# To delete the pod
kubectl delete pod vproapp

# To check logs for troubleshooting, if unable to resolve by looking at events
# section. It will show logs of vrpoapp POD.
kubectl logs vproapp

Service:
In K8, a Service is an abstraction that defines a logical set of Pods and a policy for accessing them. We use it to expose our POD as a network service i.e. If Nginx application is running inside a POD and it wants to communicate with other PODS or external users etc. 3 Types;
- Node Port: Similar to port mapping. Mostly, for development purpose.
- Cluster IP: To expose services internally i.e. Connecting Tomcat & MySQL (No port mapping).
- Load Balancer: To expose PODS to the outside network. i.e. If we want users to access the Tomcat POD.
Each Service has a Static IP address. It also has a frontend port & backend port as well as Label Selector. (API(NODE(Service->POD)))
A Service and Pod should have the same Label & Backend port in order to work.

Creating a Service with Node Port type;

# Creating a NodePort service for Vproapp Pod.
rm pods app
cd app
vim vpro-nodeport.yaml
---
apiVersion: v1
kind: Service
metadata:
name: vpro-nodeport
spec:
ports:
#Frontend port of service, which will talk to nodePort.
- port: 9000
#nodePort is a static port used for externalcommunication.
nodePort: 30005
#Backend Port of service, must be same as the exposed port of the POD.
#Application is running on this port.
targetPort: 8080
protocol: TCP
selector:
#Label used to select Pods for routing traffic. Must match the labels from Vproapp Pod.
app: webserver
#Service type.
type: NodePort
kubectl create -f vpro-nodeport.yaml

# Add incoming traffic rule in the Security group(virtual firewall) on AWS
# in Master node and 1 of the Worker Nods's security group.
# Traffic type TCP, Port:30005, My IP

kubectl delete service vpro-nodeport

Creating a Service with Load Balancer type(ELB);

vim vpro-loadbalancer.yaml
---
apiVersion: v1
kind: Service
metadata:
name: vpro-loadbalancer
spec:
type: LoadBalancer
ports:
# Instead of a static nodePort, in LoadBalancer, we use simple port to route traffic.
- port: 2000
protocol: TCP
targetPort: 8080
selector:
app: webserver
# The External IP is the IP address of the ELB assigned by AWS. 
kubectl get svc

# To delete the serivce and pod
kubectl delete svc vpro-loadbalancer
kubectl delete pod vproapp

Comparison:
- LoadBalancer services are typically used when you need to expose services to the external world, and the cloud provider manages the external load balancer.
- NodePort services are used when you want to expose services externally, but you don't need the features of a full external load balancer. It's often used in development or testing scenarios.

ReplicaSet:
The goal is to maintain a stable set of replica PODS running at any given time. It can be done across multiple worker nodes. So, if 1 POD goes down, the service running in that POD will not be unreachable as another replica is also running. Even if a Node gets destroyed, the service will still be reachable since the replica POD is running on another worker node.

# Creating a replicaset definition
vim replicaset.yaml
---
# Newer objects like Replicaset & Deployment use versioning in this format.
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: tomcatreplica
labels:
app: tomcatreplica
tier: webserver
# Info about desired state.
spec:
# Desired number of replicas at any given time.
replicas: 5
selector:
matchLabels:
# Create replica's of POD with same tier label as in template (tier).
tier: webserver

# Pod template specification for the ReplicaSet.
template:
metadata:
labels:
tier: webserver
# Specification for the pods managed by the ReplicaSet.
spec:
containers:
- name: tomcat
image: zzd786/d-application
kubectl create -f replicaset.yaml
kubectl get rs

# In production environment, it is recommended to edit the defenition file
# to scale up or down instead of using command line (Declarative)

# Scaling up/down using Command Line (Imperative)
kubectl scale --replicas=2 rs/tomcatreplica
# OR
kubectl edit rs tomcatreplica

# To delete the replica set
kubectl delete rs tomcatreplica

Deployment:
It provides declarative updates for Pods & Replica set. Commonly used for managing and scaling containerized applications. We can also use it to roll back to previous versions of the deployment.
Deployment definition creates a replica set which in turns manages the pods.

# Creating a Deployment definition
vim deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: backend

# Describes the desired state of the Deployment.
spec:
replicas: 2
selector:
# Specifies a label selector used to match pods controlled by this Deployment.
matchLabels:
app: backend

# Defines the pod template that the Deployment will use to create new pods.
template:
metadata:
labels:
app: backend
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
kubectl apply -f deployment.yaml

kubectl get deploy
kubectl get rs
kubectl get pods

# To find the current tag of the image (nginx:1.14.2)
kubectl describe pod nginx-deployment-cbc4f6496-q8bsm

# To update the deployment i.e. from nginx:1.14.2 to nginx:1.16.1
# In production env, recommended method is to edit the definition file instead of this.
kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1
# To check the rollout status
kubectl rollout status deployment/nginx-deployment
kubectl get deploy
kubectl get rs
kubectl get pods
kubectl describe pod nginx-deployment-77d6fcf879-j7smh

# To roll back to previous deployment state or revision
kubectl rollout undo deployment/nginx-deployment
kubectl get rs

# To delete the deployment
kubectl delete deploy nginx-deployment

Commands & Arguments:
We can pass commands & args to our container in K8 just like we did it in Dockerfile using CMD & Entrypoint. But in K8, we can’t directly interact with the container So, we have to pass these using POD definition file.

# Creat a POD definition file
vim pod-ca.yaml
---
apiVersion: v1
kind: Pod
metadata:
name: pod-ca
labels:
purpose: testing-ca
spec:
containers:
- name: test-container
image: debian
# Printenv command inside debian container will execute when the container starts.
command: ["printenv"]
# 2 arguments will be passed to the printenv command. Resulting command will be 'printenv HOSTNAME KUBERNETES_PORT'
args: ["HOSTNAME", "KUBERNETES_PORT"]
# The Pod will be restarted if the container exits with a non-zero status
restartPolicy: OnFailure
# Env is used to specify a variable, MESSAGE is the var name, hello is the value.
# We are passing our variable as an arg to the echo command.
#env:
#- name: MESSAGE
# value: "hello world"
#command: ["/bin/echo"]
#args: ["$(MESSAGE)"]
kubectl apply -f pod-ca.yaml
kubectl get pods

# To check the output of the conatiner, we have to see logs because the
# container will terminate after executing our commands
kubectl logs pod-ca

# OUTPUT
HOSTNAME->pod-ca
KUBERNETES_PORT->tcp://100.64.0.1:443

Volumes:
In K8, volumes provide a way for containers within a Pod to persist and share data. Volumes exist as an abstraction that allows containers to access storage independently of the underlying details of where that storage is coming from.

# Creating a Pod with MySQL container using hostpath volumes.
# Best practice is to use Persistant storage such as Amazon EBS in production.
# This is just for practice.
vim mysql-pod.yaml
# Instead of storing passwords as plain text, K8 Secrets should be used. 
---
apiVersion: v1
kind: Pod
metadata:
name: dbpod
spec:
containers:
- image: mysql:latest
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: secure-pass
#Mounts a volume named "dbvol" to the path "/var/lib/mysql" within the container.
volumeMounts:
- mountPath: /var/lib/mysql
name: db-vol

#Describes the volumes available to the Pod on worker nodes.
volumes:
- name: db-vol
hostPath:
path: /data
type: DirectoryOrCreate
kubectl create -f mysql-pod.yaml
kube ctl get pods
kubectl describe pod dbpod
kubectl delete pod dbpod

Config Map:
Config Maps are used to store config data in the form of Key-Value pairs keeping it separate from Pod definition file. We can set & inject variables & files in PODs using config map. By doing that, we decouple the config artifact from the container image and we can provide env variable data at run time.
i.e. In the case of MYSQL image, When I run a MySQL container, I provide configuration details, such as the database name, username, and password, either at creation time or runtime. This allows me to customize the behavior of the MySQL instance without modifying the actual MySQL image.

# Creating it through imperavtive method (Not Recommended)(Practice)
# db-config is the name of the configMap & we can use --from-literal to
# define Key-Value pairs/varibales.
kubectl create configmap db-config --from-literal=MYSQL_DATABASE=HaHaHa \
--from-literal=MYSQL_ROOT_PASSWORD=HuHuHu
# To check the config file
kubectl get cm db-config
# Same, but In Yaml Format
Kubectl get cm db-config -o yaml
kubectl describe cm db-config
# (Recommended or Best Practice)
# Now I will do it with declarative method through a YAML file.
# Creating a ConfigMap definition file
vim game-configmap.yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
name: game-configmap
data:
# property-like keys; each key maps to a simple value
player_initial_lives: "5"
ui_properties_file_name: "user-interface.properties"

# file-like keys with multiline values
game.properties: |
enemy.types=aliens,monsters
player.maximum-lives=5
user-interface.properties: |
color.good=purple
color.bad=yellow
allow.textmode=true

# Here the config Map hase 4 keys.
# First 2 are simple keys which represent simple string values.
# Last 2 keys have multiline values. We can store these values as a config
# file in a container. During the run time, that file will be read by the
# POD definition file.
# A pod definition file which will read values from ConfigMap and store them
# as a file inside a container.
---
apiVersion: v1
kind: Pod
metadata:
name: game-pod
spec:
containers:
- name: game
image: alpine
command: ["sleep", "3600"]
env:
# Define the environment variable 1
- name: PLAYER_INITIAL_LIVES # Notice that the case is different here
# from the key name in the ConfigMap.
valueFrom:
configMapKeyRef:
name: game-configmap # The ConfigMap file this value comes from.
key: player_initial_lives # The key to fetch.
# Define the environment variable 2
- name: UI_PROPERTIES_FILE_NAME
valueFrom:
configMapKeyRef:
name: game-configmap
key: ui_properties_file_name
# Mount the 'config' volume into the container at '/config', making ConfigMap data accessible
volumeMounts:
- name: config
mountPath: "/config"
readOnly: true

volumes:
# Define volumes at the Pod level, allowing data to be shared across
# containers within the Pod. Then mount them into containers inside that Pod
- name: config
configMap:
# Provide the name of the ConfigMap you want to mount.
name: game-configmap
# Define an array of keys from the ConfigMap to be mapped as files in the volume
items:
# Each key is mapped to a file with the name 'file1' inside the volume
# 'game.properties' will be accessible at '/config/file1' in the container
- key: "game.properties"
path: "file1"
# Similarly, 'user-interface.properties' will be accessible at '/config/file2'
- key: "user-interface.properties"
path: "file2"

# The multiline values from keys 'game.properties' & 'user-interface.properties'
# will be saved as a file who's name is defined in the key named "path" at the
# location /config inside the container.

# The volume is mounted into containers, providing access to the ConfigMap data as files
# Any updates to the ConfigMap are automatically reflected in the mounted volume
# The '/config' directory inside the container is used to organize the ConfigMap files
# To loginto the pod & verify the values of variables & files.
kubectl exec --stdin --tty game-pod -- /bin/sh
ls /config

# file
cat /config/file1
cat /config/file2

# variable
echo $PLAYER_INITIAL_LIVES
echo $UI_PROPERTIES_FILE_NAME

Secrets:
In K8, there are various types of secrets, and two commonly used types are Generic and DockerConfig secrets. Secrets in Kubernetes are used to store and manage sensitive information, such as passwords, API keys, or authentication tokens.

  • Generic Secrets:
    In generic secrets, Data stored in key-value pairs is encoded in base64 to provide some level of complication. But base64 is not a form of encryption.
    i.e. Using secrets is crucial for securing sensitive information. For example, when storing definition files with sensitive information on a Git repository, base64 encoding can prevent the information from being directly visible in plain text. However, access control to the repository should be carefully managed. It is used as best practice to avoid accidental exposure.
# Using Imperative method to encode a value using secret.
kubectl create secret generic db-password \
--from-literal=MYSQL_ROOT_PASSWORD=password12345 \
--from-literal=MYSQL_DATABASE=accounts

# MySQL pass & Databse name will be stored in base64 in the secrets file.
# It can easily be decoded by;
echo "base64 value in secrets file" | base64 --decode
# Using Declarative method
# Encode the username and password using base64
echo -n "password12345" | base64
cGFzc3dvcmQxMjM0NQ==
echo -n "dbname" | base 64
YWNjb3VudHM=

vim db-secrets.yaml
# Generic Secrets File
---
apiVersion: v1
kind: Secret
metadata:
name: db-secrets
data:
dbname: YWNjb3VudHM=
password: cGFzc3dvcmQxMjM0NQ==
type: Opaque
# Pod definition file to read secret keys
---
apiVersion: v1
kind: Pod
metadata:
name: db-pod
labels:
app: db
project: Iron-Man
spec:
containers:
- name: Infinty-Stone
image: mysql:latest
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: db-secrets
key: password
- name: MySQL_DATABASE
valueFrom:
secretKeyRef:
name: db-secrets
key: dbname
  • Dockerconfig Secret:
    If I want to access a private docker image on docker hub or any other images repo, i need to authenticate and for that purpose, docker config is commonly used.

To be continued!!!

--

--