Find an optimal set of nodes for a Kubernetes cluster

One of the most impactful ways to reduce spend on Kubernetes infrastructure is to make sure your clusters are optimally sized for the workloads they run. Working with many teams over the past year we have seen that it’s not so obvious how to arrive at an optimally-sized cluster, even in autoscaling environments. In response, we are publicly launching a new tool that we’ve used to help teams recently! While implementing this solution with users so far we’ve seen savings in the range of 25–60%, even having a major impact in autoscaling environments.

How it works

This new tool generates context-aware cluster sizing recommendations after analyzing Kubernetes metrics on historical workloads and cloud billing data. As a user, you only need to provide the context of the cluster, i.e. whether it is being used for development, production, or high-availability workloads. We then recommend cluster configurations, which optimize for the respective balance of cost, simplicity, and performance/headroom. Because this solution is Kubernetes native, we consider the following when providing recommendations:

• Analysis of historical pod/container resource consumption to predict future requirements
• Consideration of both resource requests and usage
• Ensuring largest workloads can be scheduled successfully
• Accounting for DaemonSet/kube-system replication requirements for each node
• Allocating headroom based on cluster context, e.g. dev, prod, high availability
• Avoiding shared core machines for production/HA environments

How to use

This tool is free to use and can be installed in minutes with Helm or a flat manifest. After installing Kubecost, then navigate to the Savings page and look for cluster right-sizing. You will see initial recommendations based on a “development” cluster context, but you can select a different cluster type to receive recommendations more customized to your situation.

The product supports clusters on both AWS and GCP today and begins collecting workload data as soon as it’s installed. Note: no data is egressed from your cluster with this product and everything runs with your infrastructure.

Working Example

The following example presents two potential strategies in a Production cluster. There is a tradeoff between cost and complexity in this scenario. The simpler of the two configurations requires only a single node pool, but the multi-pool solution offers an additional 14% savings per month by bin packing resources more efficiently than otherwise possible with only one node type. Advanced metrics are available directly in the UI to better understand the rationale for these recommendations.

Our Approach

To generate accurate recommendations, a potential cluster must be adequately-sized such that it is possible to schedule all workloads on the cluster. To assess this requirement, workloads scheduled as DaemonSets must appear on each node, making total resource allocation a function of node count. Furthermore, one large workload may not be able to fit on a fleet of small nodes, even if the aggregate amount of resources would appear adequate. We are able to use metrics generated by the core Kubecost allocation model (open source) to calculate nth-percentile total resource allocation over time, the percentage attributed to DaemonSets, and the largest single workload.

As a function of the cluster context, e.g. dev or production, we then analyze the shape of historical resource consumption and apply the appropriate headroom budget. For development clusters, we look at the trending 85th percentile consumption, for production 98th, and HA 99.9th.

Beyond these requirements, we apply a heuristic to manage the tradeoff between cost and complexity. For example, many teams would agree that, given the choice between saving 30% with a simple option and saving 31% with an utterly complex one, the simple option sounds more attractive. Therefore, we’ve chosen to start with two implementation strategies instead of one, hence the recommendations in the aforementioned examples.

You might be wondering why cluster sizing can not be solved by autoscaling. In the time we’ve spent helping teams of all kinds optimize their Kubernetes environments, we’ve found that cluster autoscaling, while useful, does not always resolve right-sizing concerns. Consider that autoscaling, even when it is working, does not allow for the possibility of adding different node pools to a cluster. It’s still easy, therefore, to end up over-provisioning if the autoscaler’s single node type is inappropriate for the resource proportions of the given workload. Over-provisioning by an autoscaler can be a painful experience, whereas using the cluster sizing recommendation puts the control and information in your hands to right-size your cluster when the time is right, without fear of runaway costs due to a single resource spike, memory leak, or ill-fitting node type.

About us

Kubecost provides real-time visibility into cost and capacity management decisions on Kubernetes. Our products provide showback/chargeback, resource optimization insights, and more for Kubernetes-centric environments.

This new tool is only the tip of the iceberg with respect to cluster right-sizing. We hope you’ll find it useful! If you have any questions or feedback, please reach out to us at any time on Slack or at team@kubecost.com.

Originally published at http://blog.kubecost.com on February 19, 2020.