From Autopilot to Standard GKE: The Key to 15x Cheaper Istio

TL;DR Istio proxy costs for 10-nodes GKE cluster reduced from $3065 to $185 per month.

GKE Autopilot is an amazing product that extremely simplifies the development for teams by making it easy to work with Kubernetes workloads without a need to worry about node pools, nodes themselves, their management, etc.

GKE Autopilot is indeed a comprehensive solution where the majority of workloads would be just perfectly fine and those that scales up and down frequently would not only be easier to manage but also way more cost-effective.

But as with the majority of solutions it always comes down to your particular use case. I am working on one where GKE Autopilot, even though is easier to manage combined with Cloud Service Mesh (managed Istio), may be a little bit heavy money-wise. So we decided to analyze how much would it save us if we migrate our workloads from the GKE Autopilot to GKE Standard while also migrating off Cloud Service Mesh to Istio Ambient per-node proxies.

Let’s talk about workloads and resources. In our case we’re running a cluster with a hundred deployments. Every deployment has from one to thirty active pods. Every pod usually runs a single container with the application code.

On average we’re running around 240 pods around the clock with some deviations when the traffic flows increase. The average pod workload requests 0.5 vCPU and 700Mi of RAM. So we’re looking at around 120 vCPU and 178 GB RAM of GKE Autopilot resource requests.

Now as mentioned before we’re running with Cloud Service Mesh which runs on top of Istio with sidecar mode. It means that for every pod in the mesh we inject an additional Istio proxy container. Istio proxy containers request 100m CPU and 128Mi RAM which is fair but while we’re running in GKE Autopilot, it enforces its own minimal requirements of 250m CPU and 512Mi RAM per container requests.

So proxy containers for our scenario add extra 60 vCPU and 120 GB RAM which is more than half of the resources the actual workloads consume. But even with lower proxy requirements it would still add up to 24 vCPU and 30 GB RAM which is quite a lot.

Talking money we’re looking at the cost of $5843 per month for the workloads.

And an extra cost of $3065 per month for the proxies (using current GKE Autopilot pricing for europe-west2 region) around 34.4% of the total cost and more than half of what workloads cost.

So what choices do we have here to reduce the cost while keeping the mesh functionality? The answer to that is Istio Ambient mode. 🔥Istio has just announced Ambient going GA from Istio version 1.24.

Istio in Ambient mode moves the proxies from pods to nodes. Ambient requires Istio CNI and ztunnel daemon sets to run on the nodes, so every new node will add at least two new containers. ztunnel container requires 200m CPU and 500Mi RAM and CNI container requires 100m CPU and 100Mi RAM.

Moving from per-pod sidecar proxies to per-node Ambient setup makes it possible to reduce the costs of our case from around $3065 per month to around $185 (when running on 10 nodes) for CNI and ztunnel. Depending on the needs you may add a couple of waypoint proxies which will add extra $20–$30 to the total.

For those using GKE Autopilot and Istio, the opportunity to cut costs could make GKE Standard an appealing option worth considering. While both GKE Autopilot and Standard provide robust and reliable solutions, the decision comes down to your project requirements and long-term goals.