Kubernetes: A way to reliable and scalable data-intensive application
By effectively using the technological developments, Data-intensive applications are pushing the boundaries of what can be done. An application becomes data-intensive if data is its primary challenge. Nowadays, many applications are data-intensive. Such applications have to:
- Store data in the database for the later use
- Cache data so that they can decrease the cost of expensive operations and increase the speed of reads
- Have a search functionality so that users can easily filter a large amount of data by keyword
- Periodically process and give insights from a large amount of data in database (batch processing)
Due to these minimum requirements and wide-ranging functionalities, building data -intensive applications is becoming a complicated task. There are many database systems, for example, with different characteristics designed to meet the demand of wide range of application requirements. Moreover, to meet all above mentioned requirements, you can not design your application with a single tool or component. You should break down the work into tasks, write an application code to assign each task to a single tool.
For instance as depicted in Figure 1 below, if you have an application-managed caching or full-text search server, your responsibility with the codes for the application becomes to keep caches and indexes in sync with the main database.
By combining these types of several components with distinct responsibilities, you create a data-intensive application that provides a service, and the interface of your service (API) hides these details from your end-users.
While designing data-intensive applications, you should always consider about the following questions:
- If some parts of your application go wrong (e.g. hardware failure which is considered a usual phenomenon in software engineering), does your application keep the data correctly and handle this failure without impacting the API performance ?
- If parts of the application become outdated, how can you update them by constantly keeping good service performance to the clients?
- If a significant increase in load is observed, how can the application handle this ?
In other words, your application should be reliable: its data system should constantly work in stable and correct conditions, do not lose or expose data outside even in the face of adversity such as hardware and software failures. Furthermore, you should have reasonable ways to deal with future data, traffic and complexity volume growth, namely scalability.
How can we achieve this scalable and reliable application? Is there any way out?
Yes, Kubernetes.
The purpose of this article is to introduce Kubernetes and show how it can be used to achieve scalable, reliable and maintainable application.
About Kubernetes
Kubernetes is an open-source platform for deployment containerised applications and managing them (№3 in Figure 3 below). It automates many of the manual processes including deploying, managing, and scaling containerised applications.
Kubernetes Architecture
The main building block of Kubernetes is a cluster of computers where a master node and worker nodes connect to each other and work as a single unit (Figure 2).
A cluster consists of a master node and worker node. A master is responsible for coordinating all activities including scheduling applications, maintaining desired state of applications, scaling applications, and rolling out new updates.
A worker node is a virtual machine. Each node has a Kubelet, which is an agent for managing the node and communicating with the Kubernetes master. A container consists of an application and is located in a pod. When an application is deployed, the master starts the application containers and schedules them to run on nodes.
Creating and running a cluster on Kubernetes
Kubernetes cluster can be created with both on cloud (e.g. GKE) and on local computer (e.g. Minikube). Minikube is a lightweight Kubernetes implementation where you can create a simple cluster with one worker node. In this article I have created a sample recorder application (github) and deployed it on Minikube by the following steps.
Step 1
Containerising an app in Docker with following Dockerfile:
# You build your docker and push it to docker hub. Therefore, you # should first open account at docker hub and insert account name # (in my case "arslanov") below:$ docker build -t arslanov/scalable_app_for_kubernetes:scalable_api .
$ docker push .
You can run the containerised app at http://0.0.0.0:8080 after running:
$ docker run -p 8080:8080 arslanov/scalable_app_for_kubernetes:scalable_apiStep 2
Install Minikube and kubectl
Step 3
Start the cluster:
# Creating cluster
$ minikube startYou can check created cluster with “kubectl cluster-info” and created node with “get nodes”
Step 4
We deploy the app with following deployment.yaml:
# Creating deployment
$ kubectl create -f api_deployment.yamlPods that are running inside Kubernetes are running on a private and isolated network. By default, they are visible from other pods and services within the same Kubernetes cluster, but not outside the network. Therefore, we create a service which allows us access to pods and our application in the container:
# Creating a service
$ kubectl create -f api_service.yaml# Launching a service: Now you can access an app from browser!
$ minikube service recorder-app
We have created a cluster and run our application in it. Now, we will explore how Kubernetes can give us reliable, scalable and maintainable application.
Reliability with Kubernetes
According to Klepmann (2017), reliability is defined as “continuing to work correctly, even when things go wrong”. These things can be referred as faults and your application should be fault-tolerant or resilient. However, we cannot decrease the probability of faults to zero (it is impossible) but we can prevent some serious faults from causing a system to completely stop serving (aka failures) to customers.
Hardware faults
Hardware faults (e.g. Hard Disk or CPU crash, faulty RAMs etc.)are referred as the main cause of system failures. In recent years, there is a significant move with different responses to create systems which can tolerate such hardware faults. One of the possible responses can be to create and keep redundant components “in stock”. When one of the working components dies, this redundant component quickly takes its place and application keeps running uninterrupted. But how can we create this redundant component and replace them with dead one when needed? Is there any technique? Yes, Kubernetes can be one of the techniques.
Kubernetes has ReplicaSet service which creates and maintains replica pods running at any given time. This service guarantees the availability of a redundant pod with an application container. If any pod dies inside the node, these replica pods have been replaced with dead ones immediately and your application continues to serve without failure.
You can easily create replicas of pods with containers by changing “replicas” section of “deployment.yaml” in Figure 4 as shown below:
Software faults
While running an application, we often face systematic software errors with the data system. As Klepmann (2017) stated, these type of faults lie dormant for a particularly long period of time until some unusual event triggers them. In these circumstances, some kind of assumptions might have been built, and software can be following these assumptions. Eventually, this assumption stops being true for some reason, and software failure occurs.There is no quick solution to the problem of systematic faults. One solution to these types of faults can be carefully thinking about assumptions and isolating the process and environment where these assumptions have been built from other environments. Kubernetes can also help here.
In Kubernetes, since applications are running in containers, each application’s running environment is predictable and isolated from other applications’ environments. Containers include software dependencies needed by the application, such as specific versions of programming language runtimes and other software libraries. In short, each application has its own consistent environment to run and does not affect other environment and libraries. This can often help in preventing software faults in Kubernetes.
Furthermore, replication feature of Kubernetes described above can also react to any faults that occurred in pods because of software failures and replace the faulty pods with healthy replicated ones in around 5seconds.
Human errors
Engineers and data scientists who build data-intensive applications can be another source of system faults. As Klepmann (2017) claimed, even though humans have the best intentions, humans are known to be unreliable.
If so, with unreliable humans, is there a way to make application reliable? Although human errors are beyond the scope of this article, I can suggest some ways such as:
- Designing a system with programming techniques such as object oriented programming with well-designed abstractions where one can easily detect bugs;
- Develop whole-system integration tests and manual tests;
- Set up logging and clear monitoring the running process;
- Most importantly, by rolling out code in a step-by-step process, we should minimise the impact of failure and allow fast recovery. In this case, only a subset of users may be impacted by the failure.
In the case of human errors, Kubernetes can help engineers by providing logs at container, node and cluster levels so that they can debug much easily. Refer to Logging Architecture in Kubernetes.
Example of Container level logging:
# Add "args" to api_deployment.yaml as follows and run:
$ kubectl apply -f example.yaml
$ kubectl log <container-name>The output becomes smth like this:
Deeper dive in scalability
Imagine your data-intensive application is working with 1000 customers and processing their data today. However, after one year, the system has grown from from 1000 to 10000 concurrent users. Can your application handle this sudden increase in load? In other words, does your application system have scalability?
Scalability can be achieved in three ways:
- Traditional scaling: switching to more a powerful machine with, for example, higher CPU and bigger RAM/SSD which can handle increased load:
- Creating virtual machines (VMs) and running the application by distributing its load across VMs. When the load reaches the limit, manually/automatically increases VMs and distributes the increased load:
- Each Virtual Machine has its own instance of an operating system, and you can run applications on it with access to memory, file systems, networking interfaces, and the other attributes. But flexibility comes with a cost. In this environment, the smallest unit of computing is a Virtual Machine. The guest OS may be large, even gigabytes in size. It can take minutes to boot up. Suppose your application is a big success. As demand for it increases, you have to scale out in units of an entire Virtual Machine with a guest operating system for each. That can mean your resource consumption grows faster than you like. Instead of getting a blank Virtual Machine, you should get access to a family of services that an application needs. Here come Containers with Kubernetes. You just Containerise applications and its dependencies with limited access to its own partition of the file system and hardware, and locate them on a VM and run in Kubernetes. There is only one operating system for all VMs and containers. If load increases, containers automatically(can also be manually) scales up and small VMs (suitable for the size of the containers) with the guest OS are created very fast and you just use only required resources:
Because of lower cost, much flexibility, and speed, the containerised deployment is often preferred. With the help of Kubernetes, we can orchestrate the running of containers and achieve scalability.
Example:
# Manually scaling your deployment
$ kubectl scale deployments/recorder-app --replicas=4#Launching autoscale
$ kubectl autoscale deployment recorder-app --cpu-percent=50 --min=1 --max=10
本稿は「『隔週で丸1日 研究開発・勉強Day』の仕組みを利用してData ScientistがKubernetesを試してみた」の記事になります。
