Istio gRPC Loadbalancing with GCP Internal LoadBalancer (ILB)

A couple weeks back I started looking at how to setup and expose an Istio service on GKE through a GCP Internal (and external) LoadBalancer. I did work a fair amount on gRPC with GKE (example at the end of this article), and my experience so far with Isito was just HTTP-based traffic. Coincidently, a different customer asked about how to setup a generic GCP-GKE Internal LoadBalancer for ingress traffic with Istio. After i got that bit working, i started tackling how to augment that with gRPC.

This article covers how to setup Istio on GKE, then expose an Internal (and External) LoadBalancer for gRPC traffic.

  • client_grpc_app (on GCEVM) --> (GCP ILB) --> Istio --> Service
  • client_grpc_app (external) --> (GCP ExternalLB) --> Istio --> Service

After setting this up, the gRPC traffic was (expectedly) automatically LoadBalanced _between_ pods in cluster. I expected this but to see Istio automatically loadbalance individual RPCs sent via one channel was really nice.

I’ve documented the steps here:

Anyway, lets get started (you can follow the git link above or to repeat here inline:

  • Install a baseline GKE+Istio

and most critically, set

--set gateways.istio-ilbgateway.enabled=true

(its not enabled by default on Istio 1.0.5 atleast)

  • Once the cluster is running (wait maybe 5mins after the last command), verify external and ILB IP addresses

What that’ll do is just give you the external and internalLB IP addresses (note them down; you’ll also find it on the GCP console as above)

  • Deploy sample application

The source code for the sample application is in the apps/ folder for this repo. Thats a simple app that creates one gRPC Channel to the server and on that one connection, sends 10 RPC requests.

kubectl apply -f all-istio.yaml 
kubectl apply -f istio-fe.yaml \
-f istio-ilbgateway-service.yaml \
-f istio-ingress-gateway.yaml \
-f istio-ingress-ilbgateway.yaml

The image you’re deploying is salrashid123/grpc_backend…you ofcourse don’t have to deploy that app as-is; you can look at the source then build and upload to your own repo!

Now that w’ere setup, lets test

  • Verify External client connectivity
docker  run --add-host$GATEWAY_IP \
-t salrashid123/grpc_backend /grpc_client \

You should see responses from different service addresses:

Each response shows the hostname/pod that handled the request. In this example, the responses come from different pods, round-robin (gRPC Loadbalancing!)

  • Verify Internal client connectivity

Ok, now we need to verify that we can connect via internal LB.

First create a GCP VM within the same VPC. SSH in and run.

Once in the VM, install Docker and run the following (remember to set the environment variable for $ILB_GATEWAY_IP

docker  run --add-host$ILB_GATEWAY_IP \
-t salrashid123/grpc_backend /grpc_client \

You should see responses from different service addresses:

If you deployed and setup Kiali, you should see traffic inbound from the ILB and external gateway:

So..thare you have it, gRPC loadbalacing from external and internal traffic to a service inside Istio. If you’re interested in generic GKE gRPC loadbalancing setups, please see the examples below.