Implementing Kubernetes: The Hidden Part of the Iceberg — Part 2

Kubernetes Scheduling and Resource Management.

A crane carrying shipping containers
Figure 1. A gantry crane — a specialized type of crane for loading and unloading container vessels.

The Kubernetes Scheduler

Roughly speaking, the Kubernetes scheduler is in charge of watching for new Pods and deciding the best/optimal Nodes to run them on, based on a myriad of criteria and real-time metrics collected from the cluster nodes and the applications. This scheduling challenge will be as straightforward or as complicated as your microservices and workloads are. Typically, on a production-grade K8S cluster, you will have many different microservices/applications, each with a different CPU, memory, networking, storage, and possible other non-implicit constraints and requirements that we will mention later. The K8S scheduler must consider those requirements and restrictions to properly decide the optimal nodes to run your applications/Pods.

  1. In Tetris, you have incoming geometrical shapes or tetrominoes that the player has to accommodate in the field/board, trying to optimize the available space; in K8S, you have incoming Pods that the scheduler has to accommodate in the existing Nodes, which have some free amount of memory and CPU each, same as Tetris, trying to optimize the available resources.
Tetris game
Tetris game
Figure 2. The Kubernetes scheduler as a Tetris player analogy
Chart showing 8 Gib of memory and 1.5 vCores of CPU
Chart showing 8 Gib of memory and 1.5 vCores of CPU
Figure 3. Java container/Pod that needs to be scheduled
Figure 4. Java standalone Pod definition without resources.

Resource Limits and Requests

Following the above-mentioned Java application example, if we want K8S to execute the application correctly, we need to instruct the scheduler on the CPU and memory it requires, given a particular workload. The container’s CPU and memory instructions for Kubernetes are provided using the resources dictionary, as shown below:

Figure 5. Single-container Pod definition with resources definition
Figure 6. Multi-container Pod definition with resources definition.
Figure 7. Multi-container Pod definition — CPU and memory resources
  1. CPU shares for the redis container will be 512 and 102 for the busybox container. Kubernetes always assign 1024 shares to every core, so:
    redis: 1024 * 0.5 cores ≅ 512
    busybox: 1024 * 0.1cores ≅ 102
  2. Redis container will be OOM killed if it tries to allocate more than 600MB of RAM, most likely making the pod fail.
  3. Redis will suffer CPU throttle if it tries to use more than 100ms of CPU in every 100ms (since we have four cores, available time would be 400ms every 100ms), causing performance degradation.
  4. Busybox container will be OOM killed if it tries to allocate more than 200MB of RAM, resulting in a failed pod.
  5. Busybox will suffer CPU throttle if it uses more than 30ms of CPU every 100ms, causing performance degradation.

How to properly setup resource request and limits?

Figure 8. Different resource requirements per application type.
  1. For applications where the memory reserved/used varies according to the workload, you should set the Memory request to the value observed when the application is during a “normal” workload of users and/or RPS and the Memory limit set to 20–30% above that value
  2. For applications where the CPU utilization is related with the workload, you should set the CPU request to the value observed when the application is during a “normal” workload of users and/or RPS and the CPU limit set to 40–50% above that value, due to the CPU being a special kind of resource, that can be compressed or throttled by the container runtime (kubelet), and can be somehowshared with other containers.
Figure 9. Possible consequences of too low or too high Requests
Figure 10. Possible consequences of too low or too high Limits

Key Takeaways

  1. Defining requests and limits in your application’s containers is hard, but necessary.
  2. Getting those values right can be a daunting task unless you rely on a proven method, like load testing.
  3. The Kubernetes Scheduler is the component in charge of determining which node is most suitable for running pods, based on the requests defined.
  4. If you set too low or too high values for the resources and limits for your Pod’s container(s) you will face potential issues like resource starvation and resource waste.

Conclusion

While your Kubernetes cluster and workloads might work fine without setting resource requests and limits, you will start facing stability issues as your teams and projects grow. Adding requests and limits to your Pods and Namespaces only takes a little extra effort, and can save you from being paged at 3:00 AM !