Event-driven autoscaling of Kubernetes workloads using KEDA

Krishnaraj Varma
May 20, 2020 · 6 min read

In my previous article, we explored how to use RabbitMQ Extension for Azure Functions. We discussed how to create an Azure Function using RabbitMQ Extension. The function gets triggered when we send a message to a RabbitMQ Queue. We also discussed how to generate a Dockerfile so that we can use it in production environments.

In this article, we will explore how to deploy this function in a Kubernetes cluster and autoscale it using KEDA.

What is KEDA?

KEDA is Kubernetes Event-Driven Autoscaling. KEDA is a lightweight component that determines when a container should be scaled. KEDA works along with Kubernetes’s Horizontal Pod Autoscaler. KEDA can scale any container in Kubernetes from 0 to n based on how many events are waiting to be processed.

KEDA can process Kubernetes Deployments and Jobs. Deployments are a common way to scale the workloads. When there is a long-running task, we have to use Kubernetes Jobs.

Some of the crucial concepts in KEDA are:

  1. Scaling Agent: The primary goal of the Scaling Agent is to activate and deactivate deployments.
  2. Metrics Server: The Metrics Server acts as a Kubernetes Metrics Server. Metrics Server sends scaling parameters like Queue Length, Number of messages in a stream to HPA so that HPA can initiate scaling.
  3. Scaler: Scalers are responsible for collecting metrics from an event source. The scale handler talks to the scaler to get the event metrics. Depends on the reported event metrics by the scaler, the scale handler scales the deployment or job. Scaler acts as a bridge between KEDA and external event sources. KEDA provides many built-in scalers. Kafka, RabbitMQ, AWS Kinesis, Azure Service Bus, NATS are some of the built-in scales.
  4. ScaledObject: ScaledObject is a Kubernetes Custom Resource Definition (CRD). We should deploy a ScaledaObject to scale a Deployment. ScaledObject defines the deployment name, minimum replicas, maximum replicas, metadata for scaler.

Deploy KEDA to the cluster

KEDA provides Helm charts to deploy to the cluster. First, we need to add the Helm repo.

helm repo add kedacore https://kedacore.github.io/charts 
helm repo update

Then we need to create a namespace (this is not a mandatory step, you can install KEDA to the default namespace). After that, we can install KEDA using Helm. I suppose you are having Helm 3.

kubectl create namespace keda 
helm install keda kedacore/keda --namespace keda

Now we have deployed KEDA to our cluster. We can verify it by getting the pods.

kubectl get pods --namespace=keda

You can see the KEDA operator and metrics server pods are running. Now we need to deploy our function and enable KEDA.

Deploy Azure Function on Kubernetes and enable KEDA

To deploy our Azure Function on Kubernetes, we need to generate the YAML file with Deployment resources. We can manually create the YAML file or use the Azure Functions Core Tools to create the file. We will use the Azur Functions Core Tools to create the file. We can use the func kubernetes command to generate the YAML file. To generate the YAML file, go to the project folder, and run the following command:

func kubernetes deploy --name <name-of-function-deployment> --registry <container-registry-username> --dry-run

This command will create a YAML file and deploy it to the cluster. The name parameter specifies the name of the deployment. The registry parameter specifies the container registry. The registry parameter tells the tool to run a docker build and push the image to the registry/name. Alternatively, you can use image-name to pull an image. The registry and image-name parameters are mutually exclusive. The dry-run parameter will display the YAML. Use output redirection operator to save the content.

func kubernetes deploy --name <name-of-function-deployment> --registry <container-registry-username> --dry-run > func-deployment.yml

This command will generate a Deployment resource, ScaledObject resource, and a Secrets resource. The Secrets resource is generated using the contents in the local.settings.json file. Here is the generated resource definition.

data:
AzureWebJobsStorage: VXNlRGV2ZWxvcG1lbnRTdG9yYWdlPWZhbHNl
FUNCTIONS_WORKER_RUNTIME: ZG90bmV0
RabbitMqConnection: <base64 of RabbitMQ connection string>
apiVersion: v1
kind: Secret
metadata:
name: mqfn
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mqfn
namespace: default
labels:
app: mqfn
spec:
selector:
matchLabels:
app: mqfn
template:
metadata:
labels:
app: mqfn
spec:
containers:
- name: mqfn
image: <image name>
env:
- name: AzureFunctionsJobHost__functions__0
value: fn
envFrom:
- secretRef:
name: mqfn
---
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
name: mqfn
namespace: default
labels:
deploymentName: mqfn
spec:
scaleTargetRef:
deploymentName: mqfn
pollingInterval: 5 # Optional. Default: 30 seconds
cooldownPeriod: 30 # Optional. Default: 300 seconds
minReplicaCount: 0 # Optional, Default 0
maxReplicaCount: 30 # Optional. Default: 100
triggers:
- type: rabbitmq
metadata:
type: rabbitMQTrigger
queueName: sampleq
name: inputMessage
host: RabbitMqConnection
---

We need to make one more change manually to deploy to the cluster. We need to add a metadata queueLength. The queueLength represents the number of messages in the RabbitMQ Queue, the HPA scales pods based on this number. For example, if the queueLength is 10 and there are 30 messages in the RabbitMQ queue, the HPA will scale to 3 pods. Here is the final template.

data:
AzureWebJobsStorage: VXNlRGV2ZWxvcG1lbnRTdG9yYWdlPWZhbHNl
FUNCTIONS_WORKER_RUNTIME: ZG90bmV0
RabbitMqConnection: <base64 of RabbitMQ connection string>
apiVersion: v1
kind: Secret
metadata:
name: mqfn
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mqfn
namespace: default
labels:
app: mqfn
spec:
selector:
matchLabels:
app: mqfn
template:
metadata:
labels:
app: mqfn
spec:
containers:
- name: mqfn
image: <image name>
env:
- name: AzureFunctionsJobHost__functions__0
value: fn
envFrom:
- secretRef:
name: mqfn
---
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
name: mqfn
namespace: default
labels:
deploymentName: mqfn
spec:
scaleTargetRef:
deploymentName: mqfn
pollingInterval: 5 # Optional. Default: 30 seconds
cooldownPeriod: 30 # Optional. Default: 300 seconds
minReplicaCount: 0 # Optional, Default 0
maxReplicaCount: 30 # Optional. Default: 100
triggers:
- type: rabbitmq
metadata:
type: rabbitMQTrigger
queueName: sampleq
name: inputMessage
host: RabbitMqConnection
queueLength : '20'
---

Let’s deploy the function to the cluster.

kubectl apply -f func-deployment.yml

Let’s check the deployment.

kubectl get pods -w

You cannot see the function pod since KEDA scaled it to zero.

Now let’s run the RabbitMQ producer in another command shell to send a message to the queue.

node send.js TemperatureReading 19.2

If everything goes well, you can see the function pod running.

pods creating and terminating

Now we have successfully installed KEDA and deployed a function. Since we send only one message to the RabbitMQ queue, there is only one pod. If you send around 40 messages then the HPA will create 2 pods since our queueLength is 20.

The ScaledObject

Let’s look at the ScledObject in detail and understand it. Given below is our ScaledObject.

apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
name: mqfn
namespace: default
labels:
deploymentName: mqfn
spec:
scaleTargetRef:
deploymentName: mqfn
pollingInterval: 5 # Optional. Default: 30 seconds
cooldownPeriod: 30 # Optional. Default: 300 seconds
minReplicaCount: 0 # Optional, Default 0
maxReplicaCount: 30 # Optional. Default: 100
triggers:
- type: rabbitmq
metadata:
type: rabbitMQTrigger
queueName: sampleq
name: inputMessage
host: RabbitMqConnection
queueLength : '20'

This object contains apiVersion, kind, metadata as usual. Then comes some of the interesting elements:

  1. scaleTargetRef — tells the HPA which deployment to scale.
  2. minReplicaCount — Minimum number of pods to maintain, in our case it is 0
  3. minReplicaCount — Maximum number of pods to create
  4. pollingInterval — Polling Interval, how often to contact the event source for new metrics. We should select the value judiciously, otherwise, there will be a performance hit.
  5. cooldownPeriod — Cool Down period specifies how long the system should wait before terminating the pod.
  6. type — Type specifies the scaler, in our case it is rabbitmq.
  7. metadata — Metadata is scaler specific, it will be passed on to the selected scalar.

In the next article, we will explore how to create a custom scaler.

The Startup

Get smarter at building your thing. Join The Startup’s +788K followers.

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Krishnaraj Varma

Written by

A Software Architect from Kerala, India, Open Source, Cloud Native enthusiast. Likes Golang, Rust, C/C++, Kubernetes, Kafka, etc.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +788K followers.

Krishnaraj Varma

Written by

A Software Architect from Kerala, India, Open Source, Cloud Native enthusiast. Likes Golang, Rust, C/C++, Kubernetes, Kafka, etc.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +788K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store