TCP services Load balancer for multiple Kubernetes clusters (GKE)

Published in

The Startup

7 min readMay 16, 2020

“Neither should a ship rely on one small anchor nor should life rest on a single hope”.

Well, this great quote by Epictetus teaches us a lot about reliability. And it hits the nail on the head when it comes to DevOps. Case in point: setting up multi-regional clusters to safeguard the reliability of your microservices.

You can’t rely on one Google Kubernetes Cluster (GKE) running in one zone of a region, though you can create a regional (or one case say multi-zonal) GKE cluster, where the master cluster and nodes are distributed in different zones of a region (read more about zones and regions here). That way, even if one zone is down, your services and nodes continue to run in the other zones, and you don’t have to lose any sleep over it.

But wait, what if the entire region goes down?

I’ve scoured the internet for answers, and here’s what I found… There is documentation on how to route HTTP and HTTPS traffic to multi-regional clusters: with Google Cloud Platform’s (GCP) recently introduced Ingress for Anthos, you can automatically configure Ingress using Google Cloud Load Balancer (GCLB) for multi-cluster Kubernetes Engine environments. What you’ll have is something like this:

This “Setting up a multi-cluster Ingress” document by GCP itself helps us to do this set-up and achieve High availability in case of regional failure. Simple enough, right?

But as per the Kubernetes Ingress Documentation: “Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined in the Ingress resource”. But what if I wanna route TCP traffic?

There’s also documentation on how to route TCP traffic to individual VMs.

There’s surprisingly little documentation on how to route TCP traffic to two GKE clusters (groups of VMs) sitting in different regions. If you’ve tried to look this up, you know the struggle of finding any comprehensive tutorial.

How do you route TCP traffic to multi-regional clusters?

I am going to set up highly available, multi-regional sftp servers to securely transfer files that internally use the TCP protocol, but you can do this for any TCP service.

This Setting Up TCP Proxy Load Balancing document is a great starting point. It refers only to VMs but using this as a step-by-step reference, I’ll show you how to apply it to create GKE clusters (as I’m sure you’re aware, a cluster is a group of GCP Compute Instance Virtual Machines).

Note that I’ve modified most outputs: some have been modified for security reasons, and others replaced (with dots) for conciseness.

Step 1: Configure instances spread across two regions

Instead of creating instances, we will create Clusters here. Presuming you’re familiar with the gcloud command-line tool, create two clusters in two different regions using the following commands:

# Cluster name: cluster-south
gcloud container clusters create cluster-south --num-nodes 2 --zone asia-south1-a --tags ha-cluster# Cluster name: cluster-southeast
gcloud container clusters create cluster-southeast --num-nodes 2 --zone asia-southeast1-a --tags ha-cluster

Note we have used the --tags option here, that will assign unique tags to all the nodes VMs.

gcloud container clusters get-credentials cluster-name --zone zone-name

Step 2: Create the same services on all of the instances

As the GCP document exposes a simple HTTP(S)/HTTP2 port using Apache while creating the VMs, this is an additional step in our case. So, we will use kubectl to deploy services on GKE. This kubectl tool comes preinstalled with the gcloud but if you do not have it installed already click here.

Note: We will run all the kubectl commands two times, switching the clusters to make sure the services and deployments are there on both the cluster and they are identical. To switch the context of the clusters (or to make sure kubectl is pointing to the right GKE) us the below command:

gcloud container clusters get-credentials cluster-name --zone zone-name

The below example is taken from this gist.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: sftp
spec:
  selector:
    matchLabels:
      app: sftp
  minReadySeconds: 10
  template:
    metadata:
      labels:
        environment: production
        app: sftp
      annotations:
        container.apparmor.security.beta.kubernetes.io/sftp: runtime/default
    spec:
      containers:
        -
          name: sftp
          image: 'atmoz/sftp:alpine'
          imagePullPolicy: Always
          args:
            - 'user:pass:1001:100:upload'
          ports:
            -
              containerPort: 22
          securityContext:
            capabilities:
              add:
                - SYS_ADMIN
          resources: {}

Put the above in a file named “sftp.yaml” and run kubectl apply -f sftp.yaml the below command for both the clusters.

Similarly, create a NodePort service that exposes port 30061 by putting the following manifest in an “sftp-service.yaml” file and run kubectl apply -f sftp-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: sftp-service
spec:
  type: NodePort
  ports:
    -
      protocol: TCP
      port: 80
      targetPort: 22
      nodePort: 30061
  selector:
    app: sftp

Next, run kubectl get services,pods to make sure the pods and services run on both the clusters. For both clusters, the output will be something like this:

NAME             READY   STATUS    RESTARTS   AGE
pod/sftp-hfs8v   1/1     Running   0          78s
pod/sftp-nqhb2   1/1     Running   0          78sNAME                   TYPE        CLUSTER-IP     PORT(S)
service/kubernetes     ClusterIP   14.52.232.1    443/TCP       service/sftp-service   NodePort    13.57.246.42   80:30061/TCP

Step 3: Create an instance group for each zone and add instances

You don’t need to create instance groups or add instances here, because GKE creates the group of instances for you. However, you will need a named-port on both groups. The gcloud compute instance-groups list should show the following output:

NAME                    LOCATION            MANAGED   INSTANCEScluster-south..grp      asia-south1-a        YES        2
cluster-southeast..grp  asia-southeast1-a    YES        2

Set the named-ports for both groups by running the following command:

gcloud compute instance-groups set-named-ports instance-group-name --named-ports np30061:30061 --zone zone-name

Step 4: Configure the load balancer

This step requires a series of sub-steps:

4.a: Create a health check

The health check is used by the backend services of the load balancer to see which cluster/region/service is healthy to forward the traffic. To create a health check for port 30061, run the following command:

gcloud compute health-checks create tcp my-tcp-health-check --port 30061

4.b: Create a backend service and add instance groups to it

Simply create a backend service that uses the health check and port 30061 you just created.

gcloud compute backend-services create my-tcp-backend-service \
--global \
--protocol TCP \
--health-checks my-tcp-health-check \
--timeout 5m \
--port-name np30061

Add the instance groups created by GKE to this backend service. For both instance groups, run this command one by one by one for all the groups:

gcloud compute backend-services add-backend my-tcp-backend-service \
--global \
--instance-group instance-group-name \
--instance-group-zone zone-name \
--balancing-mode UTILIZATION \
--max-utilization 0.8

4.c: Configure a target TCP proxy

Run this command to configure the TCP proxy:

gcloud compute target-tcp-proxies create my-tcp-lb-target-proxy \
--backend-service my-tcp-backend-service \
--proxy-header NONE

4.d: Reserve global static IPv4 addresses

The GCP official documentation for TCP load balancer says you can use either IPv4 or IPv6 addresses. We’ll use IPv4. To reserve a static IPv4 address, run the following command:

gcloud compute addresses create tcp-lb-static-ipv4 \
--ip-version=IPV4 \
--global

Get the IPv4 address using gcloud compute addresses list command.

4.e: Configure global forwarding rules for the address

Use the IPv4 address (say xx.xxx.xxx.xx) to create the forwarding-rule using the below command:

gcloud beta compute forwarding-rules create tcp-lb-forwarding-rule \
--global \
--target-tcp-proxy my-tcp-lb-target-proxy \
--address xx.xxx.xxx.xx \
--ports 195

Select any of the ports supported by GCP’s TCP load balancer: 25, 43, 110, 143, 195, 443, 465, 587, 700, 993, 995, 1883, 5222. (I randomly selected port 195.)

5. Create a firewall rule for the TCP load balancer

The firewall rule will allow traffic from the load balancer and health checks. The LBs and health checks source ranges are 130.211.0.0/22,35.191.0.0/16.

Remember we have used the --tags option while creating the GKE clusters, use the tag to create the required firewall rule to open ports 195 and 30061

gcloud compute firewall-rules create allow-tcplb-and-health \
 --source-ranges 130.211.0.0/22,35.191.0.0/16 \
 --target-tags ha-cluster \
 --allow tcp:195,tcp:30061

That’s it! You’re all set. Just wait a few minutes to let the load balancer set up completely, then connect to the sftp server by running the following command:

sftp -P 195 user@xx.xxx.xxx.xx

We should successfully be able to login to our sftp servers. You can try deleting the server from one GKE cluster using kubectl delete -f sftp.yaml you’ll notice if you login again, the load balancer will route the traffic to another GKE server.

Bonus Tip: To test the setup, modify mountPath: /data/incoming in the “sftp.yaml”. When deploying in cluster-south, name it mountPath: /data/incoming-1, and for cluster-southeast name it mountPath: /data/incoming-2. That way, if you log in to the sftp server and run thels command, you will get the upload folder name and easily determine from what cluster and region the response is coming.

Note: The sftp server is an example here. If you really want to deploy sftp services, the storage will not be consistent. This happens because each sftp server is using a Persistent Volume Claim running inside its own clusters. To easily manage the synchronization of the files and storage, you should mount one common Google Cloud Storage bucket in both the servers.

It’s as simple as that. Please let me know in the comments if you get stuck somewhere, or if you can suggest a better approach!

This blog was originally posted here.

TCP services Load balancer for multiple Kubernetes clusters (GKE)

How do you route TCP traffic to multi-regional clusters?

Written by Amit Yadav