Intro
As new year 2021 is coming along, I was assigned to perform a security assessment focusing on a Kubernetes environment. I had some experience with working Kubernetes and containerized environment before, but honestly, those technologies are evolving like crazy. I believe that Kubernetes is currently the most popular and even being most updated open-source project right now!
I thought it would be a good time to share some resources and techniques that I have learned about Kubernetes (or K8S). I would not go into nitty-gritty about Kubernetes technology itself since there are plethora resources to learn about it like Kube Academy.
Play with Kubernetes
Today, I will demonstrate how to create Kubernetes cluster using free Kubernetes playground tool: Play with Kubernetes and deploy simple web application to expose it to the public Internet.
Login
To login, you need to use either github or docker account. I will use docker account to login.
This will open up another pop-up window to ask for Sign-in.
After successful login, click “Start” to start the playground lab.
Add New Instance
The session will be live for 4 hours. After that, it will close session and delete everything. We will click on “+ADD NEW INSTANCE” 3 times to create 3 instances.
- Node1 (192.168.0.8) — This will be the Kubernetes Mater/Control Plane node.
- Node2 (192.168.0.7) — Worker Node1
- Node3 (192.168.0.6) — Worker Node2
Initialize Master Node
This lab is awesome that it is already installed with many of the software like kubectl and kubeadm, and it also provide initialization command to setup Kubernetes cluster.
In the Node1 (Master Node — 192.168.0.8), run the following command to initiate the Kubernetes cluster:
kubeadm init --apiserver-advertise-address $(hostname -i) --pod-network-cidr 10.5.0.0/16
At the end of the output, make sure to copy the following commands, especially for the kubeadm join command:
Note: This will be used to join worker nodes to the master node later.
To start using your cluster, you need to run the following as a regular user:mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/configAlternatively, if you are the root user, you can run:export KUBECONFIG=/etc/kubernetes/admin.conf…snip…Then you can join any number of worker nodes by running the following on each as root:kubeadm join 192.168.0.8:6443 --token cqrinh.hlgvpsp7ikqa828j \
--discovery-token-ca-cert-hash sha256:c7b9773c55b7e35a9dda67612bff1129475b36017089c3905383c04d5b43a05d
Initialize Networking
When we type kubectl get nodes command, we can see that STATUS of the node is NotReady. This is because we have not initialized networking for the cluster. Run the following command to initialize the networking:
kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml
And if we run the kubectl get nodes command again, we can see that the STATUS is now changed to Ready.
Add Worker Nodes
Now, let’s add Node2 and Node3 to the master node by using the following command on the each instance:
kubeadm join 192.168.0.8:6443 --token cqrinh.hlgvpsp7ikqa828j --discovery-token-ca-cert-hash sha256:c7b9773c55b7e35a9dda67612bff1129475b36017089c3905383c04d5b43a05d
To verify, run the kubectl get nodes -o wide command on the master node, and we can see that Node2 and Node3 are successfully joined to the master node.
Install Web Application
We will use the following nginx-app.yaml file to install a web application (Nginx) to our Kubernetes cluster:
apiVersion: v1
kind: Service
metadata:
name: my-nginx-svc
labels:
app: nginx
spec:
type: LoadBalancer
ports:
- port: 80
selector:
app: nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80[Source]: https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx-app.yaml
Without going into too much detail, the above manifest will basically create a LoadBalancer Service to expose the web app using the master node’s public IP (if the node is configured as a public-facing cluster) and deploy 3 replicas of Nginx servers.
Let’s use the following command to deploy the YAML file:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx-app.yaml
To verify, run the following command to check the created Nginx server pods in the worker nodes:
kubectl get pods -o wide3 replicas of the Nginx servers are successfully created. (2 in Node2 and 1 in Node3)
And run the following command to check the LoadBalancer creation:
kubectl get svc -o wideWe can see the created the LoadBalancer service, and it is using the port 30736/TCP to expose the Nginx server to the public.
Note: Kubernetes service will pick random ports between 30000 and 32767 by default unless a port is specified in the deployment script.
Accessing the Web App
Now, using the public URL for the lab instance + using the port 30736, let’s visit the Nginx site via browser.
Conclusion
Using the “Play with Kubernetes” lab, I demonstrated the quick setup of Kubernetes cluster + running web application within. This was really simple exercise, and it is not even 1/1000 of what Kubernetes can do. However, I think it is a great start to get your feet wet for Kubernetes at least. Cheers!
