Fight The Hidden Cost of Regional Kubernetes Clusters — Cross Zonal Egress — Part 2

Photo by Jordan Harrison on Unsplash

Part 1: https://medium.com/google-cloud/fight-the-hidden-cost-of-regional-kubernetes-clusters-cross-zonal-egress-part-1-1ee90a62c64a

In Part 1 we covered Topology Aware Routing, a Kubernetes native feature which leverages enpointslices controller to set zonal routing hints to prefer traffic within the same zone.

However, we also saw that there were many considerations and safeguards in place which may lead to the feature being disabled in your cluster.

Thankfully, like most things Kubernetes networking related, it’s Service Mesh to the rescue!!!

Istio, being one of the most feature rich service meshes, alongwith it’s advanced L7 traffic management, has capabilities to do locality aware routing which allows us to send traffic to the pod closest to the originating pod which save cross zonal costs and obviously improve latency.

In this blog, I won’t be covering installing and configuring Istio and deploying applications on your Kubernetes Clusters. You should already have the following ready:

1. A Regional Cluster with Istio installed
2. Microservices installed with istio sidecar injected

Locality Failover Load Balancing

This is the default configuration. Basically, this means that istio will prioritise sending traffic to the geographically closest pod. And as per the rules you define, if that pod is unavailable, it will route traffic to the failover zone/region configured.

This can be done by applying a destination rule like:

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: backend-dr
spec:
  host: backend-svc
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
      localityLbSetting:
        enabled: true
        failover:
          - from: us-central1-a
            to: us-central1-b
    outlierDetection:
      consecutive5xxErrors: 1
      interval: 1s
      baseEjectionTime: 1m

Here, we have configured failover to happen from us-central1-a to us-central1-b. The outlier detection is important here for failover. This is the same block used to configure circuit breaking in Istio as well.

Locality Weighted Load Balancing

There is another mode of Locality Load Balancing which is weighted distribution of traffic across your zones. In this case, you specify a percentage weight of traffic that shall be sent to each zone.

I haven’t personally come across this use case in my experience but a use case that I can think of is if a majority of your traffic is originating from one zone or if there is a noisy tenant. You don’t want one of your zones to get overwhelmed. Especially since Kubernetes scheduler doesn’t schedule pods based on incoming traffic.

To implement locality weighted load balancing, configure your destination rule like following

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: backedn-dr
spec:
  host: backend-svc
  trafficPolicy:
    loadBalancer:
      localityLbSetting:
        enabled: true
        distribute:
        - from: us-central1/us-central1-a/*
          to:
            "us-central1/us-central1-a/*": 70
            "us-central1/us-central1-b/*": 20
            "us-central1/us-central1-c/*": 10
    outlierDetection:
      consecutive5xxErrors: 100
      interval: 1s
      baseEjectionTime: 1m

In the above destination rule, we have defined weights for the traffic originating from us-central1-a.

Conclusion

In the 1st part, we saw how to use Kubernetes native contruct of topology aware routing which makes use of iptables weighting to ensure traffic doesn't;t cross zonal boundaries. However, there were many drawbacks and considerations to using that strategy.

In this part, we saw that if you’re already using Istio as a service mesh within your cluster, you can use its advanced traffic management capabilities to achieve this seamlessly. it also provides you with mcuh greater control over traffic distribution as compared to topology aware routing

Hope this helps and thanks for reading!!!!

References:

Locality Load Balancing — Official Documentation