Create Grafana Dashboards With Python

Automate your dashboards creation with grafanalib for Python

For those who do not know — Grafana is a tool to visualize your metrics/logs from multiple data sources in easy way by dashboards. For more info about its capabilities read here. Creating dashboards is easy but monotonic and boring in the long term when you have to manually click every setting for every graph. We are going to make this process less schematic.

Standard dashboard creation

Grafana dashboards are provided in json format. Most of us create dashboards and then paste it into a folder where we keep our dashboards or into configMaps if you are using prometheus-operator like me.

Json model which you copy after doing some tweaks to your dashboard

This copy-paste process of developing dashboard makes each new dashboard request unwanted. You can often make a mistake trying to manually change some value in json. How to make this process more engaging? Less painful and less “jsonful”? Add more clarity to versioning?

For purposes of the article, we will work with prometheus datasource and prometheus-operator. You can suit the presented solution to your monitoring stack with small adjustments.

Welcome grafanalib

Grafanalib is an open-source Python library in which we code dashboards. Coded dashboards are then generated in ready to deploy json format. Let’s start with installation:

Get a basic dashboard written in python with:

It is important now to check two places in example-core.dashboard.py :

1. dataSource: name of your datasource in grafana
2. expr: query expression

Adjust fields if needed. Then use binary which comes with grafanalib to create json:

Now you have a simple dashboard! I use prometheus-operator so I need to do extra steps. I create configMap containing json with specified labels and namespace, and save it to yaml file. Saving to yaml instead of pushing into a cluster is important because we use GitOps to continuous deployment. The problem here is how to add labels with create command? Luckily it is possible with this command:

Flag --local allows adding labels to conifgMap without need to communicate with API. After committing changes to Github new dashboard is ready to use!

Basic dashboard generated with Python and grafanalib

Can we make this more readable?

I thought that there is potential to make this less tight. Create functions which will return Graphs for example. My thoughts resulted in code:

Simple idea. Last print is because generate-dashboard does not allow to add multiple files without dashboard in them. However, I was disappointed — output form to_json_data() returned invalid json:

Tired 3 json validators — all of them found problems.

Read Update 08.01.2021 below — way to generate valid json

Back to first idea

I tried to validate and format output but there were so many errors I gave up. The only way to get correct json is to use generate-dashboard binary. So we are back to place when we can use only one file. If we put the function inside — then it will work — but produce a lot of redundant code with each new dashboard (unless you keep everything in one file). Another solution can be joining to grafanalib project and finding a way to fix the problem which I found.

After all dashboard definition without functions is quite good. You still have better readability than raw json. Grafanalib comes with readthedocs.io page so it is quite easy to find everything you need. Try it out, maybe you will like it.

Update 08.01.2021 — way to generate valid json

To generate valid json use functions provided in grafanalib._gen . Example:

from grafanalib._gen import print_dashboard
... 
print_dashboard(dashboard)

This will produce valid json file. Credits goes to Povilas Balciunas, he mentioned this in comments. Thanks!