CA UIM Integration with Grafana

Using Simple JSON Datasource.

We are using several kinds of monitoring tools to complete our mission in IT Operation. It covers server performance monitoring, application performance, and log management. Those metrics collector are running independently and stored in a separate database system. Hence, a different user interface. Being on the environment like this seems to cost us. We spend unnecessary time to move from one dashboard to another, just to analyze the information. It needs to reform, and currently, Grafana is the answer.

One visualization for all

One of the reasons why we’re using Grafana as a visualization tool is to display our key performance metrics on a single platform. Regardless, various monitoring system behind it. So, the ops team doesn’t have to navigate to all different monitoring tools separately. Grafana shows a promising integration capability on their data source list. Numerous sophisticated tools I know can be found here.

and there’s a lot more, as you may already know

One thing that challenges us at that moment is how we transfer server performance data to Grafana. We’re using CA Unified Infrastructure Management to collect server metrics. Unfortunately, no data source plugin available for this integration.

Adapter for our custom plugin

At first, I thought that the only way to do this is by developing our own data source plugin. On my optimistic mind, this can be done. Indeed, it is. Then I realize that I am too much ignorance about the fact that I didn’t have much time to deliver this in a short amount of time. There are always some other works to do, waiting to be finished. So, we explored again to find another approach; then we come up with this solution.

basically, it translates a request and response format

I found out that Grafana provides a data source plugin that sends http request to your custom endpoints. It’s called a simple JSON data source plugin. Http request specification for this plugin can be found here.

Grafana will call these endpoints

Grafana request and response format

At that time, we already have scripts to collect metrics data from UIM using its Rest API. So, to make it visible on Grafana, one more step needs to be done. That is, translating Grafana request format into UIM request format. Then, do the other way around; convert the results given by UIM to JSON format which grafana can read. The logic of the scripts mostly contains parsing one data input to another format.

It’s time to code

sketch as a guideline to the team

When sharing the design, I already wrote a skeleton script for this adapter, and editing query builder to match with UIM data format. So, the rest of the team can write other functions by following the guideline. The adapter itself was written in Go. Looking at the plan that looks promising, we start to code on the next day.

Finally, in a short amount of time, it can be presented beautifully on our beloved dashboard. Plus, the colors and graph design is way much better than CA UIM dashboard. In my opinion, at least.

tweak a little on plugin’s query builder

finally, it shows

write our name on a thumbnail, to send a message for the future developer here, that we exist:)

Being able to complete this task give me an insight into how this dashboard can collect information from any sources by adapting the input and output format. You just need a small effort to code in any programming language you fancy, translating the data format, then it’s good to go. It may not be the best approach, but it solves the problem we are facing.

[UPDATE]
It seems like a lot of people are searching for CA UIM Grafana plugin. The code that is described in this article is now full of private internal logic. So, I try to rewrite the code and manage to make one without using simplejson adapter.

Here’s the link to the code,

hirzani/ca-uim-datasource

Just download the package and put it in your plugins folder. Sorry if the code is not clean. It may need some improvement. However, it works and I hope it helps.