Securing Kubernetes Cluster: A Guide to API Server Connection with OpenSSL and kubeConfig

Please do support me by following me in medium , and may be through buying me a coffee, thanks.

phanindra_sangers

PreRequisites:

1. To demonstrate the working session we need a managed kubernetes cluster like EKS,AKS,GKS or ubuntu machines where kubernetes is installed using kubeadm

2. We will bootstrap the cluster using kubeadm in the ubuntu 22LTS

3. Here I am choosing AWS Cloud provider to launch the ubuntu machines.

4. And here we are using only master node we are not using any worker node

Chapter 1 : BootStrap Kubernetes Cluster Using Kubeadm

1.1 Configure CRI for kubernetes

Running this following commands

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

modprobe overlay
modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sysctl --system

apt-get install -y containerd
mkdir -p /etc/containerd

containerd config default > /etc/containerd/config.toml
vim  /etc/containerd/config.toml

##change SystemdCgroup=true

save the file

Verification:

systemctl restart containerd
systemctl status containerd

Kernel Parameter Configuration:

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sudo sysctl --system

Configuring Repo and Installation:

sudo apt-get update

sudo apt-get install -y apt-transport-https ca-certificates curl gpg

sudo mkdir -p -m 755 /etc/apt/keyrings

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

# This overwrites any existing configuration in /etc/apt/sources.list.d/kubernetes.list
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

sudo systemctl enable --now kubelet

Initialize Cluster with kubeadm (Only master node)

 kubeadm init --pod-network-cidr=10.244.0.0/16 --kubernetes-version=1.30.0

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install network addon :

Chapter 2 : User access for API Server

Creating Certificate for John

mkdir /root/certificates
cd /root/certificates

openssl genrsa -out john.key 2048
openssl req -new -key john.key -subj "/CN=john/O=developers" -out john.csr

Create CertificateSigningRequest

cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: john
spec:
  groups:
  - system:authenticated
  request: $(cat john.csr | base64 | tr -d '\n')
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
EOF

kubectl certificate approve john
kubectl get csr john -o jsonpath='{.status.certificate}' | base64 --decode > john.crt

Create John User and Move Certificates:

useradd -m john -s /bin/bash
cp john.crt john.key /home/john
cp /etc/kubernetes/pki/ca.crt /home/john
chown -R john.john /home/john

apt install net-tools 

ifconfig

grab the IP here , Replace in the Command Below

kubectl get pods --server=https://172.31.17.100:6443 --client-certificate /home/john/john.crt --certificate-authority /home/john/ca.crt --client-key /home/john/john.key

Note: Make sure to replace the above IP Address within the — server block.

Create Kubeconfig file for John:

su - john
export SERVER_IP=IP_of_your_system

kubectl config set-cluster kubeadm \
    --certificate-authority=/home/john/ca.crt \
    --embed-certs=true \
    --server=https://${SERVER_IP}:6443 \
    --kubeconfig=john.kubeconfig
 
kubectl config set-credentials john \
    --client-certificate=john.crt \
    --client-key=john.key \
    --embed-certs=true \
    --kubeconfig=john.kubeconfig
 
kubectl config set-context default \
    --cluster=kubeadm \
    --user=john \
    --kubeconfig=john.kubeconfig

Run the above commands to generate the kubeconfig file

kubectl config use-context default --kubeconfig=john.kubeconfig
kubectl get pods --kubeconfig=john.kubeconfig
cp john.kubeconfig ~/.kube/config

you can see when you do kubectl get pods you are getting forbideen this is as expected because we didnt give Authorization yet , we only got the Authentication to the API Server

So now we will create the Authorization to the API Server using ROle and Rolebinding in kubernetes

Create RBAC Role:

Now open the Different window of the instance and login into the root user

while applying below files

vim rbac.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list","create"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create"]

kubectl apply -f rbac.yaml
kubectl get role
kubectl describe role pod-reader

Create Role Binding:

#vim rolebinding.yaml


apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User
  name: john
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Verify from user John:

su - john
kubectl get pods
kubectl run john-pod --image=nginx
kubectl exec -it john-pod -- bash

now you can see that i can access the pods in default namespace there were no authorization issues

Verify API Access Action:

kubectl auth can-i create deployments
kubectl auth can-i create pods

EndWords :

Please follow me in Medium and Linkedin