Handling CPU Limits & CPU Requests In Kubernetes: Part 2

What happens if you exceed the resources request beyond what Nodes can handle in Kubernetes?

@pramodAIML
Oct 19, 2020 · 5 min read

“Choose your resources wisely else you will be thrown out ”

This is true for Kubernetes clusters with multiple nodes too. In our part 1 of handling resources and limits in Kubernetes:

We learned

  • How to manage resource provisioning of the container in Kubernetes?
  • How to define resource requests?
  • How to define resource limits?
  • How does the scheduler handle the resource provisioning?
  • What happens if some pods or containers try to go beyond the specified resource limit defined in the pod definition?

So today we will go further to understand the few resource management scenarios given below :

  • What happens when the CPU request exceeds the available CPU resource in the given cluster node?
  • What happens when the CPU limit is not specified?
  • What happens when you do specify a CPU limit but do not specify a CPU request?
  • What is the main motivation for CPU requests and limits?

Use Case 1: What happens when the CPU request exceeds the available resource in the given cluster Node?

No matter what CPU requests or limits you have assigned in your pod/container definition, if the node, accompanying the pod/containers has no more room it will not entertain any new pods asking for the room to be accommodated.

When we specify our pod definition, we generally define, CPU requests and limits, with Containers, but it is more useful and wise to think of a Pod as having a CPU request and limit. The cumulative CPU request of the containers is what we call Pod's CPU requests, similarly, the cumulative CPU limit of all the containers is known to be the Pod’s CPU limit.

So if any new pod is required to be scheduled in the cluster Nodes, the resource requests are largely used to decide the same.

Any pod will only be scheduled to run on a Node if the Node has enough CPU resources available to satisfy the Pod's CPU request.

Let’s understand this intuition by an example :

Before We Begin:

Before we create a test pod, we need to have a Kubernetes cluster, and the kubectl command-line tool must be configured locally in your system. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:

We will create a Pod that will ask for CPU requests from the Node exceeding the Node capacity in our cluster.

Let’s create a Namespace to ensure whatever resources we are creating create in this exercise are isolated from the rest of our local minikube cluster.

So, go ahead and type this command using your terminal

$ kubectl create namespace resource-example

Output:

once you have created the namespace, using the above command, you can view all the namespaces using the below-given command

Note! ns can also be represented as namespace

$ kubectl get ns 

Time to create a pod:

where the resource request being specified by the pod’s single container is exceeding the limit of what our local minikube cluster‘s Node can serve.

Pod Definition resources-pod.yaml file :

In the above pod definition

  • We have requested for 100 CPU
  • We have set the Limits to 100 CPU
  • The args(argument)-cpus "2" tells the Container to attempt for 2 CPUs, usage

Let’s create this Pod file: Using the namespace(resource-example) we defined at the start

$ kubectl apply -f /Users/prammobibt/resouces-pod.yaml --namespace=resource-example

our test-resorce-demo pod will be created in the cluster with the namespace resource-example

Let’s see what is the status of the test-resorce-demo:

kubectl get pod test-resorce-demo --namespace=resource-example

Output : As can be seen below our pod, test-resorce-demo, is showing pending as the status

Why this happening?

The output shows that the Pod status is Pending. That is, the Pod has not been scheduled to run on any Node, and it will remain in the Pending state indefinitely. To further diagnose the issue let’s use describe command :

$ kubectl describe pod test-resorce-demo --namespace=resource-example

Outcome:

As highlighted in the result above, it is clearly showing that

“The Container cannot be scheduled because of insufficient CPU resources on the Nodes.”

Use case 2: What happens when the CPU limits are not specified?

I have personally experienced that defining resource limits against CPU or memory is always better, but that is not the norm. When you are not doing so, it is imperative to understand what can be the possible implications of the same.

So if you have ignored this step in your Pod definition, you can experience two below-mentioned situations, especially when the CPU resource being asked exceeds the Node’s resource availability.

  • As our pod’s container has not been given any upper threshold limit it could use all of the CPU resources available on the Node where it is running.
  • There can also be a situation where our Pod’s container is running in some given namespace, which has some default CPU limit. In this case, our container will automatically be assigned the default limit.

A LimitRange parameter can be used by our cluster administrator to specify a default value for the CPU limit.

What’s Next?

We will discuss the remaining use cases,

  • What happens when you do specify a CPU limit but do not specify a CPU request?
  • What is the main motivation for CPU requests and limits?

with examples, in our next part 3 of handling resources and limits in Kubernetes.

So till then. It’s time to sign-off, if you are loving my contribution, please show your support by following my profile.

See you soon ….. Thanks………….

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium