Fetch data from any API using Grafana

Photo by Nicholas Cappello from Unsplash

In the last few years, Grafana has become a fantastic tool to integrate to any observability stack, and it keeps getting better by the day.

We’ve previously talked about how we use it within Pernod Ricard to gather information in a single place, with data coming from a bunch of different Datasources (Prometheus, Loki, Cloud providers, Databases…).

It can handle a lot of panels and Datasources out of the box, which will be enough for most use cases. But sometimes, you might have a more advanced requirement, and you need to go the extra mile. Conveniently, Grafana is highly extensible using plugins. You can write your own plugins, or use any plugin made available by the community. Many of the plugins are made by Grafana Labs themselves.

In this article, we will go through a quick example of how to use a very handy plugin to fetch data from any external API.

Our goal

This example will be purposely kept short and sweet, to show how Grafana can be extended beyond its usual capabilities. It will be up to the reader to adapt the idea to any case they see fit.

We will be calling Github’s public API to fetch a list of all the public repositories of a given organization. In this case, it will be our Pernod-Ricard’s very own Open Bar initiative.

Try it for yourself:

https://api.github.com/orgs/pernod-ricard-openbar/repos

Calling this URL will return a JSON payload containing all the org’s Github repos, along with a bunch of other information.

Of course, you can use any organization name instead of pernod-ricard-openbar, by changing this in the URL.

Github’s API to fetch an org’s repositories

Let’s do it !

If you want to follow along, you’ll need a running instance of Grafana, with this plugin installed : https://grafana.com/grafana/plugins/marcusolsson-json-datasource/

Thankfully, using Docker, you can spin up an instance in less than 30 seconds, just run the following command :

docker run -d -p 3000:3000 --name=grafana -e "GF_INSTALL_PLUGINS=marcusolsson-json-datasource" grafana/grafana

Note the use of the environment variable GF_INSTALL_PLUGINS with the value marcusolsson-json-datasource

This is a very practical and easy way to ship Grafana with any number of plugins you need, without having to fiddle around in the Grafana UI.

Next, login to your Grafana instance (default: admin as username and admin as password), and create a new Datasource. Choose JSON API, which is made available thanks to the environment variable we defined above.

The JSON API plugin in Grafana

The Datasource configuration can be quite advanced, you can define many authentication methods, custom HTTP headers, and forward cookies. But our example is quite straightforward, so we’ll keep it simple. Just add the Github API URL (https://api.github.com), and then “Save & Test”. If you see “Success”, you’re doing great so far.

This API is publicly available on the internet, and thus, does not require any authentication or custom headers.

JSON API plugin configuration

Next, click the “Explore” button to play around with your freshly added JSON API Datasource.

You can use the JSONPath Query syntax to parse the response and get any fields you need.

To get the full root JSON object, you can use the $ sign as follow :

JSONPath query

But in our case, we want to dig deeper, and get a list of all public repositories of the pernod-ricard-openbar organization.

Simple enough. Just head over to the “Path” tab, choose the “GET” method, and enter the following path:

/orgs/pernod-ricard-openbar/repos

Set the API path

You now have the same response that you get from this endpoint : https://api.github.com/orgs/pernod-ricard-openbar/repos

But we want to dig even deeper, and only get the data we need, mainly, the repository names. This is very easy using JSONPath.

Head back to the “Fields” tab, so you can add a filter and only get what you need.

Then type:

$.*.name

Although this expression seems quite cryptic, it’s actually pretty straightforward. It simply gets all objects inside the payload, keeps only the “name” field, and strips everything else.

And voilà, the result will look something like this :

API Call result parsed with JSONPath

We’ve been playing around inside the Explore tab so far, but of course, you can use the same idea to get any information, from any API, format it, and display it in any Grafana dashboard, along side any other Datasource.

Next steps

Really, the sky is the limit here. With this plugin you can call any API out there, public or private (using the authentication configuration of the Datasource)

Some usage examples would be:

• Get any business data from your application’s API and display it in a Grafana dashboard
• Call Cloud providers’ APIs to take any action or get any data
• Send data somewhere. This plugin also supports the POST HTTP method
• Enjoy …