A simple end to end application(db,api,ui) in local kubernetes cluster (docker desktop)

Dear readers !

I am a data engineer who is specialized in building efficient data pipelines in Azure ,Spark and python. (what is data engineering without python anyway?! :D )

Lately, I have been hearing more and more about the importance of kubernetes and why it’s a game changer. So I decided to learn the kubernetes. Luckily I have some prior experience in Dockers , Dockerfile etc. Now I need to learn as to how I can apply them in kubernetes and run them as individual light-weight applications.

I am going to assume that you know what is Dockerfile and how to build images and make containers from the images. If not, I highly suggest you to get your hands dirty on those before proceeding.

Docker overview

So what is Kubernetes ?

Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.

Overview

I have to come clean that, I am a newbie myself when it comes to kubernetes. I planned to build a simple end to end application while I am in the process of learning kubernetes.

If you are also a newbie — I highly recommend you this YouTube series on Kubernetes . It’s excellent and she (Nana) is kind enough to give us this course for free.

The above course will teach you all the components of kubernetes such as Pods,Services, PVCs etc. Hence I won’t be covering them here.

My article just explains you how to build a basic application in kubernetes with the above components.

What is my application name and what does it do ?

My application name is “Notes” and it just simply takes your notes and saves it in a database. You can add, view, edit or delete your notes anytime.

My application consists of 3 major components,

1. A Database to store the data — Here I used MongoDB.
2. An API that acts as an intermediate layer between database and front end — Here I created a simple python API with flask.
3. A simple front end — HTML file with javascript and a bit of css.

I created a simple architecture diagram for my application. Take a look and will discuss these components in details.

architecture diagram of “notes” app in k8s local

I hope you have gone through the YouTube video , that I shared earlier in this article . So I am safely assuming that you know what is Pod,Service and PVC etc.

For the sake of simplicity , I am going to keep the replica count as 1 (default) for all the Pods.

Pod1 — MongoDB

Pod Name : mongo

We are going to create a Pod which has a container that will have MongoDB database.

The above yaml file is the definition of Pod1 and it’s service.

It will do the following,

1. Creates a pod with mongodb container.
2. mongodb container is created by pulling the latest “mongo” image from DockerHub. A sign-in opearation to Dockerhub might be needed.
3. This container is accessible via port 27017.
4. It has a persistent volume claim called “mongo-pvc” which we are going to define in a moment. Pods tend to get destroyed and recreated many time. We would lose all the data in database if we don’t maintain PVC (Persistent Volume Claim) .
5. It also has a service defined — which helps us to talk to Pod1 . Accessing Pods by their direct IP address is not advised because when the pod gets recreated , it would get new IP address. Pod IP addresses are not static. Hence we talk to Pods via services which will provide us static IP address all the time.

Here is the pvc yaml file definition,

Here the volume claim is done in the host machine. In our case it’s the docker desktop vm.

Pod2 — Python API (flask )

Pod1 was created by taking the official mongo image from docker hub. However , for Pod2 , we need to create our own image with required API code ourselves. Hence we need to implement a python application with flask framework.

This is how the project structure should look like ,

project_folder
    - app.py
    - Dockerfile
    - requirements.txt

We are going to create a python application, put that application on top of a light weight linux image and create our customized image to use in our Pod2.

Let’s start coding ! (I love this part .. )

The above python application talks to Pod1. It creates new entry to Mongodb database in Pod1 , it also updates and deletes entries.

If you take a closer look in line number 8 above,

app.config["MONGO_URI"] = "mongodb://mongo:27017/dev"

“mongo” is the name of the service that points to Pod1. Pods within the same k8s cluster can communicate with each other just by using their service names. (pretty cool, isn’t it?!)

Try to spend sometime to understand the API code above. I’ll leave this to you.

Now we have the API application code ready. What’s next ?

That’s right- we need to create our image. Let’s start writing our Dockerfile,

This docker file copies our API application code to alpine image. Installs the required python modules , then runs our app.py code via port 5000.

but wait, have we created our requirements.txt file yet?

Let’s do that first ,

Now we are ready with our Python code, requirements file and Dockerfile.

Let’s build the docker file. It will create our custom python image.

Open Powershell (or CMD) , navigate to your project folder and run the following command,

docker build -t pasupathydeva/taskapp-python:latest .

This will create an image called pasupathydeva/taskapp-python in your local docker. You can verify that by using below command,

docker images

Next we are going to define our API Pod.

Important things to note here are,

1. Name of the Pod is “tasksapp”
2. The container inside the pod is exposed via port 5000.
3. Name of the docker image is : “pasupathydeva/taskapp-python:latest”
4. imagePullPolicy is set to “Never” , because the kubernetes deployment process shouldn’t look for the above image in DockerHub. This image is locally available with us. In this case, it just uses the local image.
5. Name of the service pointing to this Pod is “tasksapp-svc” and it can be reached via port 8080. (“targetPort” property is the port of the container of the Pod that it is pointing to ), and the type of the service is loadBalancer.

Pod3 — UI Application (html,javascript )

As a next step, we are going to build a simple one page UI application in html, javascript and a bit of css, then we will be putting the code on top of an nginx image to create our customized image for our UI application.

Project structure should be like this,

project_folder:
     -index.html
     -index.css
     -index.js
     -Dockerfile

Let’s take a look at our index.html file,

index.js file looks like this,

Notice the line number 1 above,

const local_host_link = "http://localhost:8080/"

It means that , the UI application hits the localhost:8080 for calling our API functionalities. Our API application is going to be exposed in localhost with 8080 port (you can confirm this by checking the “service” definition of pod2 above)

Then our index.css file should look like this,

Now let’s start creating our Dockerfile for our UI,

This docker file is pretty simple. It pulls the “alpine” image from Dockerhub and copies our project files into the image under “/usr/share/nginx/html” folder.

As usual , we are going to build this image in our local and keep it ready for kubernetes deployment. Navigate to the UI project folder in either CMD or Powershell and run the following command,

docker build -t pasupathydeva/taskappui:latest .

This will build our UI image . You can verify that by running the following command.

docker images

As a next step, let’s define our UI Pod .

Important things to note here are,

1. Name of the Pod is “tasksappui”
2. The container inside the pod is exposed via port 80.
3. Name of the image for the container is : “pasupathydeva/tasksappui:latest”
4. imagePullPolicy is set to “Never” as our image is available in local and not in Dockerhub.
5. Name of the service pointing to this Pod is “tasksappui-svc” and it can be reached via port 80 of the localhost.

Kubernetes deployment

Now that we have all of our files , we are ready to deploy those pods in our local kubernetes . Here I am using the Kubernetes cluster context of Docker desktop. Optionally you can use other kubernetes cluster contexts such as “minikube” .

Let’s make sure , you have all these files with you.

kubernetes_deployment_folder
     - mongo.yaml
     - mongo-pvc.yaml
     - tasksapp.yaml
     - tasksappui.yaml

We can deploy our Pods one after another, (using either CMD or Powershell)

Just make sure , Kubernetes is up and running in your docker desktop before proceeding,

Navigate to the kubernets deployment folder , and run the following commands .

• PVC deployment

kubectl apply -f mongo-pvc.yaml

• mongodb pod deployment

kubectl apply -f mongo.yaml

• API Pod deployment

kubectl apply -f tasksapp.yaml

• UI Pod deployment

kubectl apply -f tasksappui.yaml

You should see the “created” message after each command which confirms the successful creation of Pods.

Now, let’s take a look of all of our Pods ,

kubectl get pods -o wide

Also, let’s check if all of our services are up and running,

It looks great. Now let’s see if we can access our UI application. Go to any one of your favorite browsers , and hit http://localhost:80

Now you will see empty table because we haven’t added any data yet. Let’s go on and add couple of records.

Remember , we can access our API directly from browser as well without UI, you can hit http://localhost:8080

If you want to see all the notes , you can go to “notes” endpoint. You can see different endpoints for create, delete, update operations in the app.py code.

So , you can access the APIs by either UI application or browser.

Does our PVC work as it is supposed work ?

As I stated earlier in this article , PVC’s help to retain the data even if pods gets destroyed and recreated multiple times. Let’s see if it actually retains our data.

Let’s delete our mongo db pod.

kubectl delete pod mongo-869f6488c8-pvs87

here mongo-869f6488c8-pvs87 is the name of our mongo db pod.

Once you delete this Pod, Kubernetes is smart enough to recreate this Pod immediately. You can confirm this by running this command,

Now let’s check our UI application again, (refresh the page )

As you can see , we didn’t lose the data because of PVC. If it weren’t for PVC, we would be seeing empty table here as all the data in mongodb would have gone when the mongodb pod was deleted.

If you notice, I haven’t done anything in terms of authentication or security as I consider this entire process as a training workshop to understand important k8s components such as Pods,Services,PVCs etc.

Hope this helps you to understand the basics of K8S.

You can find all of these source codes in my git repo here.

This concludes my article , best of luck. Never stop learning !