Getting Jaeger’s Java Client internal metrics into Prometheus

We recently integrated Micrometer into the internal metrics collection mechanism for the Jaeger Java Client, making it easier to get them into Prometheus.

Image for post
Image for post
Prometheus showing the internal metrics from the Jaeger Java Client

The Jaeger Java Client, like other Jaeger clients, already had an internal metrics mechanism for collecting data such as “number of spans started”. With the support for Micrometer included in the client version 0.25.0, it’s now easier to get this data fed into a range of backend metrics platforms supported by Micrometer, like Prometheus, JMX and/or StatsD, among others.

To demonstrate this feature, we developed a simple Vert.x application that would just accept a request on a given port, create a span, write back a “Hello” message and finish the span.

Our example makes use of the Prometheus Registry as the concrete backend for Micrometer, available under the artifact coordinates io.micrometer:micrometer-registry-prometheus.

Vert.x application demonstrating the Micrometer integration with Jaeger Java Client internal metrics

For reference, this is the build.gradle file for this project. It’s pretty much the same as the starter project for Vert.x, with the addition of Jaeger and Micrometer libraries.

Gradle configuration file for our example

Run the example above with ./gradlew run and it should print out a log entry like this:

13:45:47.634 [vert.x-eventloop-thread-0] WARN  io.vertx.starter.MainVerticle - Registered tracer: GlobalTracer{Tracer(...)}
INFO: Succeeded in deploying verticle

After that, just hit the port 8080to create spans:

$ curl http://localhost:8080
Hello Vert.x!

The metrics collected are exposed to Prometheus on port 8081:

$ curl http://localhost:8081
# HELP jaeger:sampler_updates_total
# TYPE jaeger:sampler_updates_total counter
jaeger:sampler_updates_total{result=”err”,} 0.0
jaeger:sampler_updates_total{result=”ok”,} 0.0
# HELP jaeger:baggage_restrictions_updates_total
# TYPE jaeger:baggage_restrictions_updates_total counter
jaeger:baggage_restrictions_updates_total{result=”err”,} 0.0
jaeger:baggage_restrictions_updates_total{result=”ok”,} 0.0
# HELP jaeger:span_context_decoding_errors_total
# TYPE jaeger:span_context_decoding_errors_total counter
jaeger:span_context_decoding_errors_total 0.0
# HELP jaeger:sampler_queries_total
# TYPE jaeger:sampler_queries_total counter
jaeger:sampler_queries_total{result=”ok”,} 0.0
jaeger:sampler_queries_total{result=”err”,} 0.0
# HELP jaeger:baggage_updates_total
# TYPE jaeger:baggage_updates_total counter
jaeger:baggage_updates_total{result=”err”,} 0.0
jaeger:baggage_updates_total{result=”ok”,} 0.0
# HELP jaeger:baggage_truncations_total
# TYPE jaeger:baggage_truncations_total counter
jaeger:baggage_truncations_total 0.0
# HELP jaeger:traces_total
# TYPE jaeger:traces_total counter
jaeger:traces_total{sampled=”n”,state=”started”,} 0.0
jaeger:traces_total{sampled=”n”,state=”joined”,} 0.0
jaeger:traces_total{sampled=”y”,state=”started”,} 3.0
jaeger:traces_total{sampled=”y”,state=”joined”,} 0.0
# HELP jaeger:reporter_spans_total
# TYPE jaeger:reporter_spans_total counter
jaeger:reporter_spans_total{result=”dropped”,} 0.0
jaeger:reporter_spans_total{result=”err”,} 0.0
jaeger:reporter_spans_total{result=”ok”,} 0.0
# HELP jaeger:started_spans_total
# TYPE jaeger:started_spans_total counter
jaeger:started_spans_total{sampled=”y”,} 3.0
jaeger:started_spans_total{sampled=”n”,} 0.0
# HELP jaeger:finished_spans_total
# TYPE jaeger:finished_spans_total counter
jaeger:finished_spans_total 3.0

At this point, you just need to get Prometheus to scrape the port 8081. A minimalist configuration file could look like this:

Simplistic Prometheus YAML configuration

Then, start Prometheus using this file:

$ prometheus --config.file=/tmp/prometheus.yml

Prometheus should be available at http://0.0.0.0:9090 , ready to display the metrics reported by the Jaeger Client Java.

Summary

JaegerTracing

Open source distributed tracing platform at Cloud Native…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store