Deploying RabbitMQ on Kubernetes using RabbitMQ Cluster Operator

RabbitMQ up and running on Kubernetes and test using a simple Golang application.

Well, the word RabbitMQ is quite heard across the DevOps Era. So before we proceed to understand what RabbitMQ is let us understand what a messaging broker is. A message broker is a software that enables applications, services to communicate with each other and exchange data. Some examples of these messaging broker software are

1. Apache Kafka
2. Amazon MQ
3. Oracle Message Broker
4. Apache Active MQ
5. Rabbit MQ ( We will deep dive about Rabbit MQ in this article )

RabbitMQ is an open-source message-broker. It is the most widely deployed open-source message broker. RabbitMQ is lightweight and easy to deploy on-premises and in the cloud. Lets us look at the various features that RabbitMQ has to offer as mentioned on the official page of RabbitMQ

1. Supports multiple messaging Protocols like ( AMQP, STOMP, MQTT, etc ).
2. Distributed Development for high availability and throughput.
3. Supports multiple tools and Plugins.
4. An Interactive UI and an HTTP-API for managing and monitoring.
5. Rich support for a variety of languages like ( Python, Java, Go, Ruby, C#, etc ).

Image Credits: RabbitMQ

What is the entire story all about? (TLDR)

1. Install RabbitMQ operator on Kubernetes.
2. Create a RabbitmqCluster using the CRD ( Custom Resource Definition ).
3. Deploy a simple Hello World application written in Golang using a Helm Chart.
4. Test the setup by sending messages to the queue and receiving them from a client.

Prerequisites

1. A Kubernetes cluster ( Can be either On-Prem, AKS, EKS, GKE, Kind ).
2. Helm 3 installed in Your Kubernetes Cluster ( Installation Guide: https://helm.sh/docs/intro/install/ ).

Story Resources

1. GitHub Link: https://github.com/pavan-kumar-99/medium-manifests
2. GitHub Branch: rabbitmq

Install RabbitMQ operator on Kubernetes

###Install the RabbitMQ operator 
kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
###Check if the components are healthy in the rabbitmq-system namespace
kubectl get all -o wide -n rabbitmq-system

Once the rabbitmq-cluster-operator is healthy and all the related components are created let us now proceed with the installation of a RabbitMQ Cluster in our Kubernetes. Let us now clone my repo for the manifests.

$ git clone https://github.com/pavan-kumar-99/medium-manifests.git 
-b rabbitmq 
$ cd medium-manifests/ 
$ kubectl apply -f rabbitmqcluster.yaml 

Let us understand more about the manifest file that we used to create a RabbitmqCluster.

1. kind: RabbitmqCluster ( The CRD which got installed by the RabbitMQ cluster Operator. )
2. metadata.name: The name of the RabbitmqCluster.
3. spec.replicas: The number of RabbitMQ replicas to create a RabbitMQ Cluster.
4. resources.requests / resources.limits: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
5. rabbitmq.additionalConfig: The additional configuration for rabbitmq clusters. ( Reference: https://www.rabbitmq.com/configure.html#config-file )
6. service.type: The Kubernetes service type that is used to expose the RabbitMQ cluster. ( Here we are using the type LoadBalancer ).

Let us now check the status of our RabbitmqCluster.

$ kubectl describe RabbitmqCluster production-rabbitmqcluster 

Status of the RabbitMQ cluster

Now, in the above RabbitMQ Cluster manifest, we have specified the default username and password to be guest/guest. However, the RabbiMQ Cluster operator also creates a Kubernetes secret for us so that the username and the password can be extracted from there as well.

###Username
$ kubectl get secret production-rabbitmqcluster-default-user -o jsonpath='{.data.username}' | base64 --decode
###Password
$ kubectl get secret production-rabbitmqcluster-default-user -o jsonpath='{.data.password}' | base64 --decode

Now let us explore the resources created by the RabbitMqCluster

$ kubectl get all -l app.kubernetes.io/part-of=rabbitmq

Rabbitmq resources

You should now see a stateful set ( RabbitMQ always uses stateful sets to create a cluster ), headless service ( for the cluster nodes discovery ), And a Kubernetes LoadBalancer ( To access the RabbitMQ WebUI ).

Replication / Cluster Management

Now that we have seen all the components created by RabbitMQ, let us understand some important aspects of the RabbitMQ cluster. Let us check the status of the cluster using ( You can exec into any of the stateful set pod )

$ kubectl exec production-rabbitmqcluster-server-0 -- /bin/sh -c "rabbitmqctl cluster_status --formatter json" | jq

RabbitMQ Cluster status

Let us now check the nodes in the cluster.

$ kubectl exec production-rabbitmqcluster-server-0 -- /bin/sh -c "rabbitmqctl cluster_status --formatter json" | jq -r .running_nodes

Cluster Nodes

Alright, enough of the black and green terminal. Let us navigate to something colorful now. RabbitMQ has a Cluster Management Web UI exposed at port 15672. Let us get the IP of the LoadBalancer by

$ kubectl get svc production-rabbitmqcluster -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
192.168.29.166 ( Since I am using a local cluster my LoadBalancer Ip start with 192* ) 
###Let us access the WEBUI from http://192.168.29.166:15672 and login using guest/guest. 

RabbitMQ Web UI

Deploy a simple Hello World application in Golang.

To test this setup, I have written a simple Golang application, Dockerized it, and have pushed the image to my DockerHub Registry.

Docker Hub

You might be wondering why the size of the images is so small. I have reduced the size by using Multi-Stage builds in Docker. All the code and Docker files for this setup are already in the GitHub repo. Let us now start by deploying the send microservice helm chart ( This will create a WebUI for us to post the messages ).

$ git clone https://github.com/pavan-kumar-99/medium-manifests.git 
-b rabbitmq
$ cd medium-manifests/
$ helm install send-service rabbitmq-helm/send-service --set configmap.rmq_url="amqp://guest:guest@192.168.29.165:5672/"

Now that you have installed the send-service you should find the Deployment, Service ( type: LoadBalancer ), and a Horizontal Pod Autoscaler, and a configmap of the RabbitMQ URL is created and mounted as environment variables to the pod. Let us now navigate to the send-service WEB UI at port 8080. Please ignore my really bad HTML skills.

$ kubectl get svc send-service-rabbitmq-helm -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
192.168.29.166 

send-service Web UI

Let us type a message in the message section and see what happens.

send-service logs

As soon as I send my message, my send-service successfully sends the message to the RabbitMQ queue. Let us verify the same from the RabbiMQ UI.

Message from UI

The message is seen from the RabbitMQ console

Let us now install the receive-service microservice that would receive the messages from the RabbitMQ server.

$ git clone https://github.com/pavan-kumar-99/medium-manifests.git 
-b rabbitmq
$ cd medium-manifests/
$ helm install receive-service rabbitmq-helm/receive-service --set configmap.rmq_url="amqp://guest:guest@192.168.29.165:5672/"

Receiver Service logs

You should now see the messages being read by the receive microservice.

Alright, let us now send some bulk requests to see if our microservices are working as expected.

$ git clone https://github.com/pavan-kumar-99/medium-manifests.git 
-b rabbitmq
$ cd medium-manifests/
$ sh test_rmq.sh ( This will send 10 curl requests with the iterator as the message ). 

Send Service Logs

Receive service Logs

Finally Deployed...

This is a really basic scenario. RabbitMQ supports many other scenarios like Complex Work Queues, Selective Routing, Publish / Subscribe, etc. There might be many other complex use cases and scenarios. Please feel free to post them in the comment section.

Until next time……….

Conclusion

Thanks for reading my article. Hope you have liked it. Here are some of my other articles that may interest you.

Recommended

Introduction to Jenkins Operator

Introduction to Crossplane

Introduction to Bitnami Sealed Secrets

Introduction to External DNS in Kubernetes

Reference

RabbitMQ Cluster Operator for Kubernetes