Grafana Dashboard For Monitoring Debezium MySQL Connector

Bhuvanesh
Bhuvanesh
Jan 7 · 3 min read

Debezium has packed with monitoring metrics as well. We just need to consume and expose it to the Prometheus. A lot of use of useful metrics are available in Debezium. But unfortunately, we didn’t find any Grafana dashboards to visualizing the Debezium metrics. So we built a dashboard and share it with the Debezium community. Still, a few things need to improve, but almost all the metrics are covered in one single dashboard.

Debezium MySQL monitoring metrics:

Debezium MySQL connector has three types of metrics.

  1. Schema History — Track the schema level changes.
  2. Snapshot — Track the progress about the snapshot.
  3. Binlog — Real-time reading binlog events.

Setup Monitoring for MySQL connector:

We need to install JMX exporter for monitoring the debezium MySQL connector. We have already blogged about this with detailed steps.

Grafana Dashboard:

You can get this dashboard from the Grafana website. Please follow the below steps to add this dashboard to your Grafana.

  1. Go to the Grafana dashboard and click on Import.

2. In the text box type the grafana dashboard ID to import directly from the grafana website.

3. From the options, set the dashboard name and select your data source. In our case its Prometheus.

How to Monitor:

Variables:

  1. MySQL Node — MySQL Node name where the connector is reading the binlog.
  2. Connector Node — IP of the debezium connector node. It’ll be helpful if you are running the debezium in a distributed way.
  3. context — It has 2 options.
    1. binlog — If you are looking realtime binlog related metrics, then select this.
    2. snapshot — To view the snapshot related graphs, you should select this option.
  4. Plugin — Right now this dashboard only supports MySQL, we are planning to add more plugins to this.

Metrics Types:

We separated the metrics into 3 parts.

  1. Binlog Metrics — Ongoing CDC related data. (select binlog from the Context)
  2. Snapshot Metrics— Metrics about the snapshot. (select snapshot from the context)
  3. Schema History Metrics — Track the changes from the Schema level.

Screenshots:

Improve this dashboard:

Still, there are some glitches, we are working for solving them. Thanks to Sangeetha Lad who contributed to build this dashboard. So we always welcome the community to contribute to this dashboard. (it’ll be available in official Debezium’s example repo in a while)

Searce Engineering

We identify better ways of doing things!

Bhuvanesh

Written by

Bhuvanesh

BigData | Cloud &Database Architect | blogger thedataguy.in

Searce Engineering

We identify better ways of doing things!

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade