Monitor Health of RabbitMQ Queues
RabbitMQ is one of the most widely used open-source message brokers in the industry. When working with RabbitMQ, it’s important to continuously monitor health of the broker nodes. One possible and the most recommended way is to use RabbitMQ management console. Following command on your RabbitMQ server will allow management console to operate on.
sudo rabbitmq-plugins enable rabbitmq_management
Then access the management console UI from web browser using “https://<your IP or DNS>:15672”. Creating Queues and exchanges of different types and binding them using a binding key is all possible through the management console.
Another way of doing this is running a python script in the server in a time-based loop. RabbitMQ has APIs that provide enough information to do so.
To start install Requests; an open-source library that simplifies working with HTTP requests. Requests let you to send HTTP methods PUT, DELETE, GET and other requests easily. In the python script, import requests and use the RabbitMQ API method as follows.
import requests
from requests.compat import urljoin
from requests.auth import HTTPBasicAuth
uri=’http://<IP or DNS>:15672/api/queues'
resp = requests.get(uri, auth=HTTPBasicAuth(‘usename’, ‘password’))
queues = resp.json()
/api/queues gives information related to each queue as a JSON object. Exchanges, binding-key information also can be obtained by replacing queue with what is needed.
This returns a large JSON file including all details related to each queue. Here the username and password of authentication is one of the admin privileged user in your RabbitMQ deployment. Here I’m posting a part of JSON file to be clear of what information it returns.
[{‘arguments’: {‘x-queue-type’: ‘quorum’}, ‘auto_delete’: False, ‘consumer_capacity’: 0, ‘consumer_utilisation’: 0, ‘consumers’: 0, ‘durable’: True, ‘effective_policy_definition’: {}, ‘exclusive’: False, ‘garbage_collection’: {‘fullsweep_after’: 65535, ‘max_heap_size’: 0, ‘min_bin_vheap_size’: 46422, ‘min_heap_size’: 233, ‘minor_gcs’: 564}, ‘leader’: ‘rabbit@cluster1’, ‘members’: [‘rabbit@cluster1’, ‘rabbit@cluster3’, ‘rabbit@cluster2’], ‘memory’: 143204, ‘message_bytes’: 0, ‘message_bytes_persistent’: 0, ‘message_bytes_ram’: 0, ‘message_bytes_ready’: 0, ‘message_bytes_unacknowledged’: 0, ‘messages’: 0, ‘messages_details’: {‘rate’: 0.0},
………………………………
The important information here I wanted in my application were the total number of nodes in each quorum queue, leader node and available(running) nodes. Rest of the code was made to filter out the necessary parts from this JSON file.
Here I’m retrieving information of one of my quorum queues in the specific vhost ‘cmq’. It checks on the leader node, number of online members and total members so by comparison we can calculate how many failure nodes exists and whether the leader has changed.
This way looping over each queue ( use a while True with time.sleep() to loop over ), information can be retrieved and used as intended. The results can be used to continuously update a text file or create an alert to let admin knows when a node failure occurs.
Good Luck on your RabbitMQ health monitoring!
