Planning a Production-Ready Kubernetes with Fundamental Controllers & Operators — Part 5: Scheduling Workloads
TL;DR A brief overview of scheduling in Kubernetes, “the almighty” controller loop, and additional “influencers” that enhance workload management.
Introduction
In the realm of distributed systems, running additional applications should ideally be a straightforward task. If you can run a specific process on a Kubernetes cluster, theoretically, you can run any process.
However, the differentiation between systems lies in their behavior and the architectural choices made by their managers. To make any system cost-effective, it’s crucial to address the question of resource consumption, particularly the “four primitives” — CPU, RAM, network, and disk.
For stateless applications, focus primarily on CPU and memory requests. For stateful applications, additional constraints like disk speed, network bandwidth, or GPU capabilities may come into play.
Let’s begin with resource requests and limits followed by a node-selection based on tolerations.
Resource Management in Kubernetes
Effective resource management in Kubernetes is vital, particularly for CPU and memory. Setting appropriate resource requests and limits during pod creation ensures that applications have the necessary resources to operate smoothly.
Pod Resource Requests and Limits
Here’s an example of a simple pod configuration for a web application, illustrating resource requests and limits in the pod’s manifest file
apiVersion: v1
kind: Pod
metadata:
name: web-app
spec:
containers:
- name: web-container
image: nginx:latest
resources:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
- Requests: The pod requests
64Mi
ofmemory
and250m
(0.25 cores
) ofCPU
, guaranteeing these resources for the pod. - Limits: The pod sets a limit of
128Mi
ofmemory
and500m
(0.5 cores
) ofCPU
, representing the maximum resources the pod can use. Exceeding these limits may result in throttling or eviction.
Scheduler Considerations
When the kube-scheduler
attempts to place this pod, it looks for a node that can accommodate at least the requested resources. The scheduler ensures that the pod has the necessary resources to start and run effectively, considering existing resource usage on nodes to avoid overcommitting resources.
Avoiding Resource Limits
Many community leaders advise against setting resource limits due to potential issues. For further insights, refer to Robusta.dev’s blog and Spot.io’s guide on CPU limits.
Vertical Scaling
At this point we either have nodes
or node-pools
which means we have a predefined set of resources to use in our cluster.
Unless you have a restricting policy in place (another option to keep your cluster tidy — for a different post), your users may not set limits
/ requests
on their workload and “someone needs” to provide some kind of “guess-timation”, in this case we use VPA.
Vertical Pod Autoscaler (VPA)
Vertical Pod Autoscaler (VPA) frees users from the necessity of setting up-to-date resource limits
and requests
for the containers in their pods.
When configured, it will set the requests automatically based on usage and thus allow proper scheduling onto nodes so that appropriate resource amount is available for each pod.
It will also maintain ratios between limits and requests that were specified in initial containers configuration.
It can both down-scale pods that are over-requesting resources, and also up-scale pods that are under-requesting resources based on their usage over time.
Configured with a Custom Resource Definition object called VerticalPodAutoscaler. It allows to specify which pods should be vertically autoscaled as well as if/how the resource recommendations are applied.
Vertical Pod Autoscaler (VPA) for web-app
The Vertical Pod Autoscaler (VPA) automatically adjusts the CPU and memory requests for your pods based on their actual usage.
- VPA Configuration:
First, ensure that the VPA components are installed in your cluster. Then, create a VPA resource for theweb-app
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: web-app-vpa
spec:
targetRef:
apiVersion: "v1"
kind: Pod
name: web-app
updatePolicy:
updateMode: "Auto"
This controller is a good starting point for optimizing the workload on your cluster and is also used under the hood by a tool called goldilocks which can be used as a “requests recommendation engine“.
When you install goldilocks it creates the
vpa
automatically and starts monitoring the current workload in order to help predict the current usage and if you preserve history it will provide the trends of the application over time and help set the correctrequest
statement.
Once we’ve covered resource allocation on a node
, we can apply the same logic to a node-pool
and as our cluster scales with our product needs, we need to start labelling
and segregating
between certain systems based on budget/hour etc …
This is where other supporting mechanisms play a big role both in vertical and horizontal scaling which we will dive into in the next part in the series.
Supporting Mechanisms in Scheduling
While the kube-scheduler
is the primary controller for scheduling, other mechanisms influence pod placement:
- Node Controllers:
Ensure nodes are registered with the API server and have necessary labels/taints for scheduling. - Pod Affinity and Anti-affinity Rules:
Specify pod placement rules to enhance workload distribution and fault tolerance. - Node Selectors:
Use labels to control pod placement based on node attributes. - Custom Scheduler Plugins:
Extend kube-scheduler functionality with specific scheduling criteria through custom plugins.
These mechanisms work alongside kube-scheduler to achieve desired pod placement strategies within your cluster.
When you are scheduling resources the following process is performed by the scheduler:
- Resource Limits and Requests:
Ensure pods are scheduled on nodes with available required resources. - Resource Labels:
Organize and select resources efficiently using labels. - Resource Tolerations:
Allow pods to be scheduled on nodes with specific taints. - Affinity and Anti-Affinity:
Ensure certain pods are scheduled together or apart based on application needs.
Let’s see an example
Using Labels, Taints, and Tolerations
Consider a scenario with nodes designated for specific workloads. Label these nodes and use taints to ensure only appropriate pods are scheduled on them this starts with tainting the node and then scheduling the pod.
- Node Configuration
# our "tainted-node"
apiVersion: v1
kind: Node
metadata:
name: special-node
labels:
dedicated: special-workload
spec:
taints:
- key: "dedicated"
value: "special-workload"
effect: "NoSchedule"
- Pod Configuration with Tolerations
# our "special-pod"
apiVersion: v1
kind: Pod
metadata:
name: special-pod
spec:
containers:
- name: special-container
image: nginx
tolerations:
- key: "dedicated"
operator: "Equal"
value: "special-workload"
effect: "NoSchedule"
In this example:
The Node Configuration:
The node is labeled with dedicated=special-workload
and tainted with the same key-value pair and the NoSchedule
effect, which means only pods that tolerate that taint can be scheduled on it which is what we see in the Pod Configuration: which includes a toleration matching the node’s taint, allowing it to be scheduled on the designated node.
To conclude the topic of scheduling let now look at our cluster as 1 single computer which has some boundaries which are the accumulative sum of the 4 primitives available on the underlying operating systems, as we plan our clusters we may consider providing labels and taint combinations which represent your business needs and budget, what we should do when planning our environments landscape categorize the services in terms of common
/ shared
which may require their own node-pool to provide services as we discussed in previous chapters related to configuration management/ingress routing.
Managing clusters involves grouping node-pools or node-groups, indicating specific workload requirements, such as disk speed or GPU capabilities. For instance, Kubernetes often has a control plane node group with labels like role=master
or role=worker
. Adopting this pattern throughout the system, with the help of additional controllers, optimizes cluster management.
Understanding Kubernetes scheduling intricacies is just the beginning. The next logical step is exploring scaling and auto-scaling mechanisms. These concepts will help efficiently manage resources and maintain optimal performance as workloads grow.
Stay tuned for the next chapter, where we’ll delve into strategies for scaling and auto-scaling in Kubernetes, ensuring robust and flexible clusters capable of handling dynamic demands.
Yours sincerely, HP