AKS internal load balancer with static IP address that is generated dynamically

Azure Kubernetes Services(AKS) allows provisioning of both external as well as internal Load balancer. Load Balancer help in distributing traffic between multiple pods backing them. A public load balancer has a public IP address assigned to it and reachable on internet while an internal load balancer has an internal IP address associated to the Pod virtual network. It does not have public IP address associated with it. It means an internal load balancer is not reachable via internet. It is only available from within the Kubernetes cluster.

A Load Balancer is created using the Service Kind in AKS. A Service by default creates a public load balancer in AKS. Let’s look at an example of creating a public load balancer in yaml format.

apiVersion: v1
kind: Service 
metadata: 
  name: Public-IP-address
spec: 
  type: LoadBalancer 
  ports: 
  — port: 80 
  selector: 
    app: internal-app

The code before creates a new Service resource in AKS. AKS understands that the type of service is Load balancer and it provisions a new load balancer or reuses an existing external load balancer and assigns a public IP address to it. The IP address can be obtained by executing the command on CLI.

kubectl get service Public-IP-Address

There is no specific attribute within the Service definition that provisions an internal load balancer. To provision an internal load balancer, annotations can be used and this is specific to AKS. Annotations are additional metadata information that can be added to any resource. AKS uses this metadata information to provision an internal load balancer instead of public load balancer.

The next code segment creates an internal load balancer instead of a public load balancer in AKS. Note the annotations attribute — it contains information that an internal load balancer should be created by assigning a true value to it. Another thing to notice here is that a dynamic IP address from the POD subnet will be assigned to the load balancer instead of the public IP address.

apiVersion: v1 
kind: Service 
metadata:   
  name: internal-app   
  annotations: service.beta.kubernetes.io/azure-load-balancer-        internal: "true" 
spec:   
  type: LoadBalancer    
  ports:   
  - port: 80   
  selector:     
    app: internal-app

The challange with this approach of creating an internal load balancer is that it will get assigned a new internal IP address in case the service object is recreated after deletion and this might break client applications in case they are directly using this IP address. One solution to this problem is to assign a static internal IP address to the load balancer as shown next. Note the attribute loadBalancerIP with value of 10.240.0.5

apiVersion: v1 
kind: Service 
metadata:   
  name: internal-app   
  annotations: service.beta.kubernetes.io/azure-load-balancer-        internal: "true" 
spec:   
  type: LoadBalancer   
  loadBalancerIP: 10.240.0.5   
  ports:   
  - port: 80   
  selector:     
    app: internal-app

while the code with statically assigned IP address to internal load balancer works and should be fine in most of the situation, there is a better way to do so. Image what would happen if the static IP address is not available at the time of creation of load balancer? What happens when you delete the service and meanwhile someone creates a POD and get assigned the load balancer IP address.

In short, the current solution needs active administration to ensure that it keeps on working in future.

A better solution is to create an internal load balancer without any static IP address. Let Azure and Kubernetes decide the best next available IP address available from the network subnet and then somehow make the newly generated IP address static for the load balancer.

To do this, create a new Service without any static IP address as shown here again.

apiVersion: v1 
kind: Service 
metadata:   
  name: internal-app   
  annotations: service.beta.kubernetes.io/azure-load-balancer-        internal: "true" 
spec:   
  type: LoadBalancer    
  ports:   
  - port: 80   
  selector:     
    app: internal-app

after creation of internal load balancer gather the newly dynamically assigned IP address

kubectl get service internal-app -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

now this IP address can be replaced in the original code using bash utilities like sed.

The new yaml file should like shown next. Note that the IP address is dynamically generated and then put statically in the yaml file. There is no need to be find or be aware of what IP addresses are available within the network subnet.

apiVersion: v1 
kind: Service 
metadata:   
  name: internal-app   
  annotations: service.beta.kubernetes.io/azure-load-balancer-        internal: "true" 
spec:   
  type: LoadBalancer   
  loadBalancerIP: 10.240.0.5   
  ports:   
  - port: 80   
  selector:     
    app: internal-app

now, the new file can be applied again using command

kubectl apply -f ‘internal-lb.yaml’