Application Monitoring of RabbitMQ with Prometheus and Grafana

Published in

redbus India Blog

6 min readOct 20, 2023

Application Monitoring is essential for every System. In this article, we will know about the Prometheus and Grafana, and how can we use it in our Software System for monitoring and enable it with RabbitMQ.

What is Application Monitoring and why is it necessary?

Application Monitoring is the process of monitoring an application’s performance, availability, and end-user experience to ensure the application is functioning properly.

For example: No of Request Per Sec, It will tell us about how many request is coming in our system in every second.

How to create Grafana Dashboard of RabbitMQ :

RabbitMQ : RabbitMQ is a widely used open-source message broker that helps in scaling the application and easy to deploy a message queuing mechanism in between the two applications.

RabbitMQ provides our own management Web-UI in there. we can see all the activities of rabbitmq whatever happening.

Now, in new version RabbitMQ support prometheus exporter. it enable the metric details of rabbitmq which will we can use it in prometheus for monitoring purpose.

How to install and run:

You can download the rabbitmq from their website. I am sharing the details how to install and run it in docker below:

sudo docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 -p 15692:15692 rabbitmq:3-management

Note : RabbitMQ exposes the metrics on a dedicated TCP port, 15692 by default.

How to enable rabbitmq prometheus exporter

sudo docker exec -it rabbitmq bash

run the below command

rabbitmq-plugins enable rabbitmq_prometheus

To confirm rabbitmq metrics, run the curl command

curl -s localhost:15692/metrics | head -n 3

If you have installed rabbitmq in local then run the plugin command directly. RabbitMQ support cluster rabbitmq metrics also. you can check it in their website.

I am sharing the details how to install and run prometheus and grafana with rabbitmq below:

Prometheus:

Prometheus is an open-source technology designed to provide monitoring and alerting functionality for cloud-native environments, including Kubernetes. It can collect and store metrics as time-series data, recording information with a timestamp. It can also collect and record labels, which are optional key-value pairs.

Key features of Prometheus :

Multidimensional data model — Using time-series data, which is identified by metric name and key-value pairs.
PromQL — A flexible querying language that can leverage the multi-dimensional data model.
No reliance on distributed storage — All single server nodes remain autonomous.
Pull model — Prometheus can collect time-series data by actively “pulling” data over HTTP.
Pushing time-series data — Available through the use of an intermediary gateway.
Monitoring target discovery — Available through static configuration or service discovery.
Visualisation — Provides multiple types of graph and dashboard in their metric dashboard.

How does it work :

Prometheus scrapes metrics from instrumented jobs, either directly or via an intermediary push gateway for short-lived jobs. This is an intermediate server that monitoring targets can push their metrics to before exiting. The data is retained there until the Prometheus server pulls it later. It stores all scraped samples locally and runs rules over this data to either aggregate and record new time series from existing data or generate alerts.

Prometheus provides us the PromQL (Prometheus Query Language), which is help for manipulate the data which is store in prometheus.

Prometheus provide us our own web-UI where we can check all functionality of prometheus like Target , Alert , Graph etc. In Graph, prometheus provide us the dashboard where we can manipulate the data and plot graph by PromQL.

I have shared below one example of Graph in prometheus web-UI

How to install and Run:

For installing and configuring the prometheus you can follow their website, I am sharing the details how to install and run it in docker below:

Create configuration file with prometheus.yml file.

# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "rabbitmq"
    static_configs:
      - targets: ["host.docker.internal:15692"]

The config file tells Prometheus to scrape all targets every 5 seconds and targets are defined under scrape_configs and evaluation_interval option controls how often Prometheus will evaluate rules.

Use the docker run command to start the Prometheus Docker container where you have created Prometheus.yml file.

sudo docker run -d --rm -p 9090:9090 -v `pwd`/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus:v2.20.1

You can check your prometheus on localhost:9090 Prometheus Web-UI.

Grafana :

Grafana is an open-source platform for monitoring and observability that lets you visualize and explore the state of your systems. Capable of ingesting metrics from the most popular time series databases. It is not only show time series graph but also provide multiple chart graph.

Key features of Prometheus :

Explore metrics and logs
Build dashboards
Annotate dashboards
Set up alerts
Provide inbuilt datasource like Azure, Kafka, MySQL etc.

Why we need Grafana :

Now a days, Monitoring is very essential part for running multi-tier architecture and application in DevOps era. Grafana provides us alert generate part also which help us to tell about issue before too much impact in system.

We can create multiple graph in a dashboard which will help us to moritoring multiple graph at a time.

How to install and Run:

For installing and configuring the grafana, you can follow their website, I am sharing the details how to install and run it in docker below:

Create configuration file with datasource.yml file.

apiVersion: 1
datasources:
  - name: RabbitMQ Prometheus
    type: prometheus
    access: proxy
    orgId: 1
    url: "http://host.docker.internal:9090"
    basicAuth: false
    isDefault: true
    editable: true

sudo docker run -d --rm --name grafana -p 3000:3000 -e GF_AUTH_ANONYMOUS_ENABLED=true -e GF_AUTH_ANONYMOUS_ORG_ROLE=Admin -v `pwd`/datasource.yml:/etc/grafana/provisioning/datasources/datasources.yml grafana/grafana:10.1.1

You can check your grafana on localhost:3000 Grafana Web-UI.