Near real-time monitoring charts with Spring Boot Actuator, Jolokia and Grafana

Spring Boot Actuator gives a nice out-of-box way to watch for applications performance and responsiveness, by /metrics endpoints.

Therefore, in a clustered and elastic environment, applications nodes can grow, multiply and be composed by a very large number of instances. Monitoring isolated nodes can be painful and inconclusive. A tool for aggregating time-series data fits better.

This post is intended to find out a solution for monitoring Spring Boot Metrics over time, in a time-series fashion, with no implementation needed, only by tooling and configuration.

APMs like NewRelic, AppDynamics or DataDog does a great job by using JVM and bytecodes instrumentation to generate their own metrics, insights and relevant transactions. It’s possible yet to annotate methods with @Timed. Therefore, all available resources under Actuator library are ignored. Another issue by using this kind of took is related to data retention, that can obfuscate small time-window data.

NewRelic 1-minute time-window restriction on discovered and instrumented transactions

spring-boot-admin might be another option, since it connects to Spring Boot instances, aggregate nodes and so on. However, /metrics endpoints are not monitored along timeline, neither aggregated among different nodes of the same application module (horizontal scaling). It means you have both situations: no data-series metrics but only instant and isolated snapshots of Actuator data of isolated nodes.

Spring Boot Admin with metrics from Actuator: a snapshot of metrics data of a given application node
Spring Boot Admin with JMX and MBeans read data of a give application node

jconsole and visualvm might be another option, connecting directly to JMX, through RMI, to nodes. Actuator store Metrics data inside a MBean from JMX. Additionally, by using Jolokia, MBeans are exposed to a RESTful HTTP enpoint, /jolokia. So, the same information can be acquired in both endpoints: JMX MBean Metrics and Rest HTTP Jolokia endpoint. Nevertheless, the same problem resides in connecting directly to individual nodes in a clustered environment, additionally with painful RMI old-school protocol. Workarounds ahead.

JConsole old-school JMX Metrics of a given application node
VisualVM JMX Metrics of a give application node

Moving ahead, I’ve checked if it is possible to solve this issue with some Ops-team modern tools:

After some research, I ran into a better solution with InfluxDB and Telegraf and zero coding, only by configuring correct parts.

In a nutshell, configuring all those parts are quite simple:

Spring Boot Actuator Raw Metrics
Metrics sent by Telegraf to InfluxDB, collected by Jolokia and JMX over HTTP
Grafana InfluxDB data source configuration
Grafana Metric chart query and configuration: gauges of an API

That’s it. It’s all working. I’ve put an example configuration in my github project with docker-compose. Check it out!

You can reach me if you have any trouble configuring those monitoring tools.