Multizone Kubernetes and VPC Load Balancer Setup

Published in

vmacwrites

4 min readApr 27, 2021

Securely expose your Kubernetes app by setting up a Load Balancer for VPC in a different zone.

You can expose your app to the public by setting up a Kubernetes LoadBalancer service in your IBM Cloud Kubernetes Service cluster. When you expose your app, a Load Balancer for VPC that routes requests to your app is automatically created for you in your VPC outside of your cluster.

In this post, you will provision an IBM Cloud Kubernetes Service cluster spanning two private subnets (each subnet in a different zone), deploy an application using a container image stored in an IBM Cloud Container Registry and expose the app via a VPC load balancer deployed to a public subnet in a different zone. Sound complex? Don’t worry, you will provision and deploy the app using Terraform scripts.

Here’s a simple architecture diagram to show what you will achieve by following the steps in this post:

The administrator provisions everything from VPC to subnets to load balancer using Terraform scripts.
The user will access the application via the VPC load balancer URL.

Why IBM Cloud Kubernetes Service on VPC?

With IBM Cloud Kubernetes Service clusters on VPC compute, you can create your cluster on VPC infrastructure in the next generation of the IBM Cloud platform, in your Virtual Private Cloud. VPC gives you the security of a private cloud environment with the dynamic scalability of a public cloud. VPC uses the next version of IBM Cloud Kubernetes Service infrastructure providers, with a select group of v2 API, CLI and console functionality.

Prerequisites

Let’s get started

Open a terminal or command prompt on your machine, clone the GitHub repository and move to the directory:

git clone https://github.com/VidyasagarMSC/vpc-private-k8s-public-lb.git cd vpc-private-k8s-public-lb

The file structure in the GitHub repo should look like this:

2. Create the local.env file from the template file provided in the repo and update the environment variables accordingly. Once updated, source the file:

cp templates/local.env.template local.env 
source local.env

3. Execute the main.sh script that, in turn, runs the Terraform commands and setup.sh shell script:

./scripts/main.sh

The below are the resources provisioned:

A VPC
Three subnets — one in each zone:
Subnets 1 and 2 in zone 1 and zone 2 are private
subnet-lb in zone 3 is public and will be used to deploy the load balancer
A public gateway attached to subnet-lb
An IBM Cloud Kubernetes Service cluster spanning two zones of subnet 1 and subnet 2
A Kubernetes namespace (if provided as part of local.env)
A Container Registry namespace (if no registry namespace is mentioned in your local.env)

4. You should see the public load balancer URL in the output. Check the URL with the below cURL command:

curl <Load_Balancer_URL>:8080

Under the hood

If you are wondering how the load balancer is deployed to a subnet in a different zone, you are in the right section.

First, you need to check the generated file loadbalancer.yaml - specifically, the annotations. If you observe closely, this annotation - service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets - is doing the magic.

apiVersion: v1
kind: Service
metadata:
 name: myloadbalancer
 namespace: $KUBERNETES_NAMESPACE
 annotations:
   service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: public
   service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets: $SUBNET_ID
spec:
 type: LoadBalancer
 selector:
   app : $MYPROJECT-deployment
 ports:
  - name: http
    protocol: TCP
    port: 8080
    targetPort: 8080

Ensure that you add a label in the metadata section of your deployment configuration file. This custom label identifies all pods where your app runs to include them in the load balancing.

The annotation is to specify one or more subnets to which the VPC application load balancer service deploys. Note that you can specify a different subnet in the same VPC than the subnets to which your cluster is attached. In this case, even though the VPC application load balancer deploys to a different subnet in the same VPC, the VPC application load balancer can still route traffic to your worker nodes on the cluster subnets. To see subnets in all resource groups, run ibmcloud ks subnets --provider vpc-gen2 --vpc-id <vpc> --zone <zone>.

For other supported annotations, check the documentation here. The following provides additional info while using the annotations:

service.kubernetes.io/ibm-load-balancer-cloud-provider-enable-features: "proxy-protocol": Know Your Real Clients: Use PROXY Protocol on IBM Cloud Kubernetes Service
service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-node-selector: For respective key=value, run the kubectl get nodes --show-labels command

Clean-up

To clean up the setup, run the below command:

./scripts/cleanup.sh

What’s next?

Check out other Companion scripts to VPC tutorials
Simplify your Terraform scripting experience on IBM Cloud with Schematics

If you have any queries, feel free to reach out to me on Twitter or on LinkedIn.

Originally published at https://www.ibm.com on April 27, 2021.