How to calculate CPU for containers in k8s dynamically ?

It’s possible to dynamically resize CPU on containers in k8s with the feature gate “InPlacePodVerticalScaling”.

Before this feature gate, sizing CPU was error prone and, in reality, we would often put something too high, to not deal with latency.

Too much CPU and precious resources are wasted, too few CPU and the app is slowed. Let’s explore the ways to dynamically resize CPU.

Strategy 1 : always use 80% of CPU

The simplest idea. If the app is using 50% of CPU then we resize to use 80%. We keep 20% of free CPU as safety measure.

It works well when we resize down but there is an issue when we resize up. Imagine if the CPU usage is 100%, does that mean we should double the CPU or maybe just increase a bit ? There is no way to know.

Strategy 2: always use 80% of CPU with exponential increase

When we resize down, it’s the same as strategy 1.

Let’s explore the case when we need to resize up. It means the CPU usage is between 80% and 100%. We have this formula:

new_cpu = average_cpu / 0.8+ (average_cpu / 0.8 * coeff)²

So we will increase exponentially. It’s nice because we will increase a lot if we are at 100% but not that much if we are at 90% for example.

In practice, this strategy works great and is the one used by Kondense: Resources sizing tool for kubernetes.

Strategy 3: Target 100% usage by using CPU usage and pressure

When we resize down, we resize to use 100% of CPU so no resources are lost.

When we resize up, we look at CPU pressure. It’s a value on each containers in microseconds. This value tells us how many microseconds we have lost because we did not had enough CPU. The higher this value, the more we should increase CPU.

This strategy is in theory the most optimised. In reality I have not yet succeeded to make this strategy work. The main issue is that CPU pressure is sometimes quite high but increasing the CPU will not lower it depending on the application.

This strategy could beat strategy 2, I probably need more work to find the right formula that works independently of the application.

Conclusion

Dynamic resize is a big change for k8s, it’s gonna make use waste way less resources. In few months/years we will have something that will resize our containers dynamically and we will not have to worry about it anymore.

To try dynamic resize, you should check Kondense: Resources sizing tool for kubernetes.

Thanks for reading this,