Tutorial: Kubernetes 201
Introduction
At Weld North Education, Kubernetes has been a hot topic. There is an increasing demand for it as a skill set in the tech industry. I’ve been fortunate enough to have worked with Kubernetes, aka k8s, since late 2017 (shortly before attending my first KubeCon). Back then there weren’t as many tools or tutorials available as there are today. Amidst the plethora of walkthroughs, tutorials, and guides I offer one more.
This tutorial focuses on leveraging k8s in Docker Desktop to run a small system. It highlights the use of Deployments, Services, Ingresses, and Horizontal Pod Autoscalers (HPAs). This is an intermediate to advanced tutorial and will get quite detailed and involved (as it should for a ‘200-level’ tutorial). By the end of it, you should feel somewhat comfortable with writing some k8s manifests (that’s what the YAML files are called if you’re not familiar) and executing kubectl commands.
Prerequisites
Installation
- Docker Desktop (2.3.0.3) https://www.docker.com/products/docker-desktop
- Kubernetes (v1.16 recommended; comes with Docker Desktop 2.3.0.3; some older versions will work but it’s not guaranteed)
Make sure you have enough computer hardware to allocate to Docker. It can consume a lot of CPU/memory/disk space, if you allow it. For this tutorial, I recommend allocating a minimum of 1–2 cores and 2 GB of memory to Docker (it won’t use anywhere near that much). You won’t need a ton of disk space allocated to Docker for this tutorial (even if you skip to the end and are advanced enough to deploy Persistent Volumes and Persistent Volume Claims).
If you need help figuring out how to set up and install Docker Desktop and Kubernetes, there is a section in this tutorial with references to a few guides.
YAML
The majority of k8s manifests use YAML (they also support JSON but pretty much everyone uses YAML). Get very familiar and comfortable with YAML syntax. If you’re not familiar or comfortable with YAML, here are a few good resources to check out:
https://blog.stackpath.com/yaml/
https://rollout.io/blog/yaml-tutorial-everything-you-need-get-started/
If you’re feeling confident in your YAML skills but still want to learn more, check out this link:
Set up Kubernetes in Docker Desktop (10–15 min)
Refer to one of these Medium articles for a detailed guide on setting up Kubernetes in Docker Desktop (I simply Google searched installing Docker Desktop for Mac OS X and Windows 10):
https://medium.com/backbase/kubernetes-in-local-the-easy-way-f8ef2b98be68 (Mac OS X)
https://towardsdatascience.com/windows-and-docker-and-kubernetes-oh-my-2020-d21be00b168b (Windows)
Follow the detailed guide until Kubernetes is installed and set up in Docker Desktop. Don’t install the Kubernetes dashboard and don’t install Helm (regardless of whether it tells you to install them or not). This tutorial should prepare you enough that you won’t need the dashboard. You can do some unguided learning later with Helm if you want.
Once you have Docker Desktop set up and Kubernetes installed and running, you are ready to start doing some real work.
Write Pod Manifest and Deploy (5–10 min)
Pods, the Basic Unit of Kubernetes
Pods in Kubernetes are composed of containers. In this tutorial, we use Docker-based containers. Containers in a pod all have access to the same resources as all of the other containers. These common resources include mounted disks, network ports, CPU cores, and RAM. Each Pod runs on a Node. Each node is a separation of hardware and has an allocated amount of CPU cores and RAM. For the purposes of this tutorial, we will only be running 1 Node on Docker Desktop.
There are 2 types of containers that run in a Pod: init containers and containers. Init containers run in-order until completed prior to any other containers. Containers consist of the main container and side cars. Any container that is not deemed the main container is a side car.
For more detailed information on Pods, Containers, Nodes, and Init Containers refer to these links:
https://kubernetes.io/docs/concepts/workloads/pods/pod/
https://kubernetes.io/docs/concepts/containers/
https://kubernetes.io/docs/concepts/architecture/nodes/
https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
Write YAML, Deploy Pod
This is a basic Pod definition that prints out Hello World!to standard out in a busybox container. When applied to the k8s system, the Pod definition gets converted into a Kubernetes API request (with authentication).
Type out the above Pod definition save it as a file called pod.yaml (I used Visual Studio Code). Open up your favorite terminal (if you’re using Visual Studio Code, you can use the built-in terminal) and execute the command kubectl apply -f pod.yaml. This will tell the Kubernetes API to create a Pod in the default namespace called red-pod. You can see all Pods that are available in the default namespace by executing the command kubectl get pods. At first (if you execute the command quickly enough) the status of the Pod will show ContainerCreating. After a little bit of time, if you execute the command kubectl get pods again the Pod should show a status of Completed. Finally, you can see the output of standard out for the Pod by executing the command kubectl logs red-pod and it should show Hello World!.
Once you’ve completed this part of the tutorial, you can take down the Pod by executing the command kubectl delete -f pod.yaml. This will remove the Pod from the k8s system. You could also delete the Pod using the command kubectl delete pod red-pod. Both are valid and have their use case. Another valid way to delete the Pod is to use the command kubectl delete -f . but this assumes that pod.yaml is the only manifest file in the current folder.
NOTE: make sure that you use kubectl apply and not kubectl create or kubectl run. Otherwise, the Kubernetes API will reject any kubectl apply commands on the Pod YAML file after making changes to it. The apply command will also only work on any mutable fields and the Kubernetes API will tell you if something is invalid in the Pod manifest.
More Interesting with Deployments (8–15 min)
Deployments
Pods in Kubernetes can be standalone but are usually managed by a Deployment. Each Deployment in k8s keeps track of a few ReplicaSets, which in turn keep track of Pods. Each Pod in a ReplicaSet shares common configuration based on a template in its manifest.
For more information on Deployments and ReplicaSets, refer to these links:
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/
Pods are managed by other k8s constructs such as Jobs and StatefulSets but we’re not going to go over those in this tutorial. However, here are a few useful links with information on Jobs, CronJobs, StatefulSets, and DaemonSets:
https://kubernetes.io/docs/concepts/workloads/controllers/job/
https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
Write More YAML, Deploy a Deployment
The YAML for defining a Pod is simpler than for a Deployment. There are additional required fields such as a parent spec object that contains a replicas field and selector object. The replicas field defines how many Pods should run concurrently at any given time and the selector object defines how to choose which Pods will be managed by the Deployment. In this section of the tutorial, we match on the label key-value-pair app: blue. The template field is also required and is used to define Pods in the Deployment. In the above example, the template essentially mirrors the Pod definition (aside from the additional env configuration and slightly different command). The additional env field specifies any environment variables that will be configured for each Pod and in this specific example the Deployment manifest uses the Downward API to retrieve the Pod’s name as defined in the metadata section. All of this allows us to use the environment variable podname in the echo command (and then have the Pod sleep for 10,000 seconds so that the Pod doesn’t crash or terminate too quickly).
Type out the Deployment manifest and call the file deployment.yaml. Apply the manifest to the k8s system using the command kubectl apply -f deployment.yaml. You can get all Deployments available in the default namespace by executing the command kubectl get deployments. You can then execute the command kubectl get pods to see that there is a single Pod named blue-deployment-<hash>. The Deployment appends a unique hash to the name of the Deployment and uses it for the Pod names. Check the standard output of the Pod by executing the command kubectl logs blue-deployment-<hash> and it will display the name of the Pod.
This didn’t offer us much more than the Pod definition that we applied other than adding a unique hash to the name of the Pod. So, let’s scale the Deployment so that it has more than 1 Pod. Execute the command kubectl scale deployment blue-deployment --replicas=3. This will create 2 additional Pods for the Deployment. Execute the command kubectl get pods to verify that there are 3 Pods. Another way to update the Deployment is to update the manifest file by setting replicas: 3 instead of replicas: 1 and executing the command kubectl apply -f deployment.yaml. One last way is to edit the manifest directly in k8s using the command kubectl edit deployment blue-deployment. Finally, take down blue-deployment with the command kubectl delete -f deployment.yaml. This will take down the Deployment and also any associated Pods.
It’s Getting Exciting: Services (10–15 min)
Services
A service in k8s maps ports to Pods and routes traffic to them. The default behavior is to route the traffic to the Pods in a modified round-robin way. Each Pod will be cycled through and receive traffic in some arbitrary get-next-available order (but anything coming from the same client will try to be routed to the same Pod that initially handled any requests from that client). Services can be headless and be able to route to all Pods that they map via DNS (you can still route to the Pods directly but the DNS names aren’t as obvious). Headless services are typically used with StatefulSets due to the strict naming convention that StatefulSets impose. Services support a few different types of routing and load balancing. For the purpose of this tutorial, we’ll be using the default type which is ClusterIP.
For more detailed information on Services, refer to this link:
https://kubernetes.io/docs/concepts/services-networking/service/
Write Even More YAML, Deploy Service
Type out the Service manifest and save the file with the name service.yaml. Services bind to Pods in similar way that Deployments do (using a selector object). Ports can be defined with or without a name and the service listens on these ports to forward traffic to a Pod.
As it currently sits, the above Service manifest (when applied to the k8s system) won’t route any traffic since there aren’t any app: rainbow. We could either change the label to bind to app: blue but it wouldn’t be very useful since blue-deployment simply outputs the name of the Pod to standard out. We need something that can listen on a port and handle requests. Something like nginx.
The above example Deployment manifest is a little crazy. There are probably much easier ways to create a server…but that would require a different image. Type out the above Deployment manifest and save it as a file called deployment-for-service.yaml. Execute the command kubectl apply -f deployment-for-service.yaml and check the status of the Pods by executing the command kubectl get pods. Once one or all of the Pods have a status of Running, we can send traffic to one of them by executing the command kubectl port-forward rainbow-deployment-<hash> 8080:80 and then opening up a browser and using the URL http://localhost:8080 (you could also use curl or Postman or Insomnia or any other tool that can issue http requests).
Now let’s forward the port on the Service by executing the command kubectl port-forward service/rainbow-svc 8080:80. Open your browser (or tool) and use the URL http://localhost:8080 again. Most likely you got the same result as the first run when we port-forwarded a single Pod. To prove that the Service will forward traffic to any of the available Pods, delete the Pod from the k8s system that responded using the command kubectl delete pod rainbow-deployment-<hash>.
Routing Traefik (10–15 min)
Ingresses
Ingresses manage incoming traffic in the k8s system (whereas egress would manage outgoing traffic). Ingresses are tied to an ingress controller (which runs on the k8s system and handles routing of requests) and the most supported are ingress-gce and ingress-nginx. We use Traefik by Containous at Weld North Education for a few of our k8s clusters.
For more information on Ingresses and some additional Ingress controller options, refer to thses links:
https://kubernetes.io/docs/concepts/services-networking/ingress/
https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
Deploy Traefik 1.7
Traefik has released many 2.x versions but we still use 1.7 in a few of our k8s clusters at Weld North Education (due to the huge shift in setup and configuration).
There are a couple of k8s constructs and Deployment features that I gloss over in the above YAML. This tutorial won’t cover those features in-detail (ConfigMaps, securityContext field, volumeMounts object, and volumes object). Type out each of the above examples as separate files and name them traefik-configmap.yaml, traefik-svc.yaml, and traefik.yaml respectively. Apply all of them to the k8s system using the commands kubectl apply -f traefik-configmap.yaml, kubectl apply -f traefik-svc.yaml, and kubectl apply -f traefik.yaml. After waiting about 30 seconds, execute the commands kubectl get svc and kubectl get pods.
Take note of the Port(s) associated with traefik-svc (in particular the local port, which is a value in the 30000–32767 range). We’ll use this value in the next part of the tutorial.
Write and Deploy Ingress
Type out the above Ingress and save it as ingress.yaml. The annotations section under metadata tell k8s which ingress controller to use and additional routing information specific to Traefik. The rules section specifies the hostname, associated paths to route, and k8s service to route traffic. Execute the command kubectl apply -f ingress.yaml to add the Ingress to the k8s system.
In order for the next part of the tutorial to work properly, you’ll need to modify your hosts file to map 127.0.0.1 (aka localhost, but use the actual IP address) to custom-local-ingress.com.
Each platform handles the hosts file differently. Quick Google search gives this link, if you need extra guidance on modifying it:
https://support.rackspace.com/how-to/modify-your-hosts-file/
Once your hosts file has been properly modified, open a browser and use the URL http://custom-local-ingress.com:<nodeport> where nodeport is the local port that we took note of earlier in this section. Refresh a few times and see if you can get all 3 Pods to give a response.
The True Value of K8s: Autoscaling Pods (10–15 min + 15 min for scale up/down)
Horizontal Pod Autoscalers aka HPAs
Horizontal pod autoscalers (or HPA for short) trigger scaling of Pods (usually for Deployments) based on configurable criteria (most commonly based on target CPU percentage). This feature coupled with cluster-autoscalers (not covered in this tutorial) allow k8s to increase/decrease the number of Pods (and thus service instances) based on quantitative traffic patterns. There is a catch, however, and that is that autoscaling only works as well as the metric and threshold (target value) used for scaling and also how quickly the Pods initialize and become ready to respond to traffic.
For more information on HPAs, refer to this link:
https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
Deploy Metrics Server
Docker Desktop does not currently come with Metrics server installed (which is necessary in order for HPA to have metrics to use). Use the following resource to deploy Metrics server to the k8s system:
https://blog.codewithdan.com/enabling-metrics-server-for-kubernetes-on-docker-desktop/
You’ll know that you’ve configured and deployed Metrics server correctly when there are results after executing the command kubectl top pods. Each Pod running in the k8s system default namespace should show CPU and memory usage.
Write and Deploy HPA
Type out the HPA manifest and save it as hpa.yaml. Apply the HPA to the k8s system using the command kubectl apply -f hpa.yaml. Get the status of HPAs in the k8s system by executing the command kubectl get hpa.
Notice that there is still an issue and the Deployment associated with the HPA returns an unknown CPU percentage. We need to update the Deployment so that each Pod has a resources object defined with limits and requests for both CPU and memory.
After making the changes, apply the changes to the Deployment using kubectl apply -f deployment-for-service.yaml (or you can do what I did and copy the Deployment manifest, make changes, and save it as deployment-for-hpa.yaml). After applying the changes, execute the command kubectl get hpa again and it should now show a CPU percentage.
Test the Scaling Feature of the HPA
Type out this Pod manifest and save it as load-generator.yaml. Execute the command kubectl apply -f load-generator.yaml to begin applying load to the Service. Execute the command kubectl get hpa to see the percent CPU utilization vs the target. Execute the command kubectl top pods to see that the Pods are getting CPU pressure.
HPAs by default calculate the average metric value over a 5 minute window to make a scaling decision. They start to scale up pretty soon after making a scaling decision and have a 10 minute cooldown for scaling down. Execute the commands kubectl get hpa and kubectl top pods after about 5 minutes to see that the HPA has caused the Deployment to scale up.
Remove the load-generator Pod (and thus remove load from the Service) by executing the command kubectl delete -f load-generator.yaml (or you can execute kubectl get pods and delete the Pod). Execute the commands kubectl get hpa and kubectl top pods to see that there isn’t any load on the Service or Pods.
After about 10 minutes, execute the commands again.
Final Remarks
After successfully completing this tutorial, you should feel somewhat comfortable with k8s manifests, deployment, and monitoring. All of the YAML used for this tutorial can be found here:
https://github.com/ImagineLearning/kubernetes-docker-desktop
This is the end of the guided portion of this tutorial. If you’d like some unguided help (and a challenge) then here are a few suggestions:
- Learn more about ConfigMaps and use one for nginx.conf instead of just an environment variable
- Do something more with the nginx containers than simply returning the Pod names
- Learn more about volumes and volume mounting for Pods and mount the nginx.conf directly instead of using a shell script
- Persist data across Pod restarts by using a Persistent Volume and Persistent Volume Claim
- Learn more about Secrets and use one as an environment variable in one of the Deployments
- Mount nginx.conf from a Secret
