Create a Kubernetes Cluster Using Docker Desktop

Creating multiple pods based on multiple images

Prerequisites:

• Docker Desktop installed
• Basic Kubernetes knowledge
• Code editor

Objectives:

• Enable Kubernetes on Docker Desktop
• Create Redis YAML file to deploy 4 pods
• Create Apache YAML file to deploy 10 pods
• Deploy Redis and Apache pods using the YAML files
• Communicate between pods

Introduction

Kubernetes is a highly complex container orchestration tool that allows you to automate software deployment, scaling, and management. It is like Docker Compose on steroids and comes with way more specification options, which can make it a bit more difficult to learn and use. In this tutorial, we will create a Kubernetes cluster and deploy 4 Redis pods and 10 Apache pods. Before we get started, there are some important key words that you should familiarize yourself with:

Control Plane

The control plane consists of master nodes that are making decisions. The control plane must have at least one master node, but can have many more. Inside the master nodes, you will find the API server, scheduler, controller manager, etcd, and more. Here are a few key terms:

• kube-apiserver — frontend for the Kubernetes control plain
• scheduler — controls how and where your containers are placed on the nodes in objects called pods
• controller manager — uses the API to determine the state of the cluster and make sure it is in the state that the YAML specifications define; responsible for maintaining the specified number of replicas and restarting pods when they have failed
• etcd — database backend that is a distributed file system of key values
• coreDNS — controls DNS records

6 node cluster (3 masters, 3 workers)

A node is a single server (instance) in a K8s cluster. Note that Kubernetes on Docker Desktop is not configurable and only supports a single-node cluster. The services that we create in Docker Desktop (4 Redis, 10 Apache) will all be running on the master (control plane) node. Ideally you would want a master node with two worker nodes, but this is not possible with Docker Desktop. Despite that, this tutorial will be useful to show you how to build your Kubernetes YAML file.

A pod is the basic unit of deployment in Kubernetes. It is like a container for your containers. You can have multiple containers within a pod and multiple pods within a node.

A cluster is a set of nodes, but can also be a single node. A single-node cluster would consist of just the control plane. A multi-node cluster would consist of any number of master nodes and any number of worker nodes. The diagram above is an example of a multi-node cluster.

Kubectl is the CLI to configure K8s and manage apps and is automatically installed with Docker Desktop.

Kubelet is the K8s agent running on nodes. It is the API engine that talks to the local runtime and back to the control plane. Kubelet is automatically installed with Docker Desktop as well.

Kube-proxy controls the network and maintains network rules on nodes.

Now that you know some of the basic terminology, let’s get started with Docker Desktop. If you already have Docker Desktop installed, skip the next section.

Install Docker Desktop

To install Docker Desktop, follow these instructions:

Docker Desktop for Windows

Docker Desktop for Mac

Docker Desktop for Linux

Enable Kubernetes on Docker Desktop

To enable Kubernetes on Docker Desktop, first open Docker Desktop and click on the Settings icon. On the left side menu select Kubernetes, then click Enable Kubernetes. On the bottom right, click Apply & restart (mine is greyed out because I’ve already completed these steps). You are now ready to start using Kubernetes in your CLI.

Kubernetes Objects

Kubernetes objects are constant or persistent entities that you create in the Kubernetes system. Examples of objects include pods, services, volumes, namespaces, deployments, replica sets, jobs, and more. Objects are defined in a YAML or JSON file and later created from this file. For this tutorial, I will use YAML. Each file contains one or more resource descriptions, and these descriptions are called manifests. Each manifest describes one of the objects. Each manifest has at least four required entries, and these include:

apiVersion:
kind:
metadata:
  name: 
spec:

apiVersion: — Which Kubernetes API version you want to use

kind: — What type or kind of object you want to create

metadata: — Data to uniquely identify the object

name: — Required metadata in string format

spec: — Desired state for the object

API Version, Kind, and Metadata

How do we know what options are available to nest under each of these required entries when creating our YAML file? We can use the following command to get a list of all of the resources that a cluster supports and the API version it needs.

kubectl api-resources

You will get back an extensive list of objects that you can create. Notice the list includes the API version that you will need to indicate in your YAML file, as well as the Kind name. API versions differ depending on what kind of object you are creating. For example, if you want to create a Deployment, you will need to use apps/v1 for the API version and Deployment for the Kind in your YAML file.

Some common Kind objects are Pod, Deployment, and Service. A Pod Kind deploys a single pod. For this tutorial, we want to deploy 4 Redis pods and 10 Apache pods so we will use the Deployment Kind. Within the Deployment Kind, we will be able to specify the number of replicas we want.

The metadata field is required and must include a specified name. To see all of the key options for metadata, use the following command:

kubectl explain deployments --recursive

Metadata options

Spec

The spec is where you define all of the details of the objects you want to create. We know that we will be using the Deployment Kind for creating our pods with multiple replicas. There are tons of options to include with spec, and you can see all of the different available resource keys that the Deployment Kind supports by using the the same command in the last section:

kubectl explain deployments --recursive

Sample spec options

There are many more deployments spec options, but too many to capture in a single screenshot.

To get an explanation of each spec option to determine if you want to include it in your YAML file, use the following command:

kubectl explain deployments.spec

When building your YAML file, you are basically nesting options. You can use the above command to see what options are available. You can define your options further by adding .<option> as many times as you want, down the option tree. This is a bit hard to explain, but here is an example. If I want to see the options for deployments.spec, I run the above command and am given the below output.

This tells me all the options I can nest under deployment.spec in my YAML file. I know I want to use replicas because I want to create 4 replicas of Redis and 10 replicas of Apache. It is best practice to put each of these services in their own pods, so we will be creating two YAML files. Let’s get started on the Redis file.

First create a new directory where you want to house your YAML files. Then create a new file (I’ve named mine k8s_redis.yml). Open the file in the CLI via VIM or in your favorite text editor. I am using VS Code.

Earlier we found the API version for Deployment and noted it to be apps/V1. The kind is Deployment. For metadata, a name is required, and we will call this pod redis-deploy. We are going to use some optional labels, which you can learn about here. Name is the only requirement for metadata, and you cannot use underscore (_) in names.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: redis-deploy
    labels:
        app: redis
spec: 

Now let’s get into Spec. This is where we will define all of the details of the objects we want to create. Using the screenshot above, we know that we will want to include replicas to indicate the number of replicas we want, selector to use with our labels, and template to describe the pods that will be created. Let’s add those to our YAML file, nested under spec.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: redis-deploy
    labels:
        app: redis
spec:
    replicas:
    selector:
    template:

Add Replicas to Redis YAML File

Replicas is probably one of the easiest to get correct. Simply add the number of replicas that you want to create. We want 4 replicas of Redis.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: redis-deploy
    labels:
        app: redis
spec:
    replicas: 4
    selector:
    template:

Add Label Selector to Redis YAML File

Labels are key: value pairs that are attached to objects (like pods or deployments). Labels are optional but are good practice to include in case you need to search for a specific pod later. A label selector can help a user identify a set of objects.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: redis-deploy
    labels:
        app: redis
spec:
    replicas: 4
    selector:
        matchLabels:
            app: redis
    template:

Add Template to Redis YAML File

Template is where the meat is. This is where we define our container within our pod. To see what options are available, use the following command:

kubectl explain deployments.spec.template

You can see that our only options here are metadata and spec.

Since we’re using labels, we will need to include that under metadata. To see what options will be available to nest under metadata, add the .metadata extension to the previous command. There are a lot of options for metadata so I have included a screenshot of just a few.

If you scroll through metadata, you will notice that labels is an option. Let’s go ahead and starting adding to our YAML file.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: redis-deploy
    labels:
        app: redis
spec:
    replicas: 4
    selector:
        matchLabels:
            app: redis
    template:
        metadata:
            labels:

To see what options we can nest under labels, add the .labels to the previous command. Hopefully this is starting to make sense how you can determine options by .<option> to your explain command.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: redis-deploy
    labels:
        app: redis
spec:
    replicas: 4
    selector:
        matchLabels:
            app: redis
    template:
        metadata:
            labels:
                app: redis

After adding our metadata, we want to start adding the spec information for the template. Use the following command to see the spec options to nest under template:

kubectl explain deployments.spec.template.spec

There are a ton of options, and we will use containers to define the container we want to create. Here is the completed Redis YAML file:

apiVersion: apps/v1
kind: Deployment
metadata:
    name: redis-deploy
    labels:
        app: redis
spec:
    replicas: 4
    selector:
        matchLabels:
            app: redis
    template:
        metadata:
            labels:
                app: redis
        spec:
            containers:
            - name: redis
              image: redis:7.0.9
              ports:
              - containerPort: 6379

Use the same tree extension method we used above to explain options available. Note that under containers, you do need - before name and containerPort. Indentation is key in YAML. You can have two or four spaces for nesting. Note that where you include - it is directly under the above word, with a total indentation of two instead of four. In the containers section, you give your container a name and identify the image name and version, which can be obtained from Docker Hub. You can optionally identify ports to expose on the pod’s IP address.

Create Apache YAML File

To create the Apache (httpd) YAML file, use the same steps and techniques that you used to create the Redis file. First create a new file in the same directory as the previous file, and name it something like k8s_httpd.yml. The apiVersion and kind are the same. Change the metadata labels to reflect httpd. Change the replicas to 10. Change the name of the container and the image to reflect the httpd image and version you want to use. Change the containerPort to 80.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: httpd-deploy
    labels:
        app: httpd
spec:
    replicas: 10
    selector:
        matchLabels:
            app: httpd
    template:
        metadata:
            labels:
                app: httpd
        spec:
            containers:
            - name: httpd
              image: httpd:2.4.56
              ports:
              - containerPort: 80

Deploy Pods

To create the pods from our YAML files, we will use the following command:

kubectl apply -f <file_name.yml>

We will have to run this command twice (once to create the Redis pods and once to create the Apache pods).

Create Redis pods

Create Apache pods

To verify everything you just created, use the following command:

kubectl get all

You should see 4 Redis and 10 Apache pods. You can also see the cluster IP that was automatically assigned to the cluster and the ReplicaSets that were created when you defined replicas in your YAML files.

Confirm Cluster Creation

After running the kubectl get all command, we have evidene that our cluster was created. Remeber, Docker Desktop can only create a single-node cluster, so everything is running on the master node (control plane). To verifty this, use the following command:

kubectl get nodes

The above command outputs a list of nodes in your cluster. We can see from our output that we only have a single node running, and that node is the control plan or master node.

Communicate Between Pods

In Kubernetes, containers within a pod can automatically communicate with each other across localhost. If you want a pod to be able to communicate with another you can do this using each pod’s unique IP address. You can also create a service, but this is out of the scope of this tutorial. To get the IP address of pods, use the following command:

kubectl get pod -o wide

To test communication between pods using their IP addresses, we can use the ping command. Use the following command to enter into a pod. I will be entering my first Redis pod (redis-deploy-cdd55dd6d-hrmfh).

kubectl exec --stdin --tty <pod_name> -- /bin/bash

The /bin/bash opens a shell script so you can run commands. Run:

apt update -y

Then run:

apt upgrade -y

To install ping, run:

apt install iputils-ping -y

Now that ping is installed, we can ping any of the other pod IP addresses to test reachability. Use control C to stop the ping.

Clean Up

To delete the pods you created, first get out of the pod shell using the exit command. Then use the following command:

kubectl delete deployment <deployment_name>

Conclusion

Thank you so much for following along with me and reading this tutorial. Kubernetes is a whole world of complexity and this tutorial just scratches the surface. For more information, visit the official Kubernetes documentation. Until we meet again, keep learning and keep pushing yourself!