Let’s monitor our Spring microservices with Prometheus and Grafana?

In this article I will try to briefly explain how we should apply observability in our applications.

Grafana dashboard

1. To get started, we must include the Actuator libraries in our API.

Follow the Spring Actuador documentation:

https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuato

Add Actuator dependency to pom.xml

#To add the actuator to a Maven-based project, add the following “Starter” dependency:

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
</dependencies>

With the library added to the project, we will be able to access the application’s metrics by accessing the documentation produced by Actuator. Accessing at: http://localhost:8080/actuator

We will have some standard endpoints with some metrics for this application.

To expose some more metrics options, we must add a configuration to the Application properties:

Then we restart the application and we will have more endpoints exposed in Actuator:

To have details of the requests, we access the metrics:

http://localhost:8080/actuator/metrics/http.server.requests

From the list of several related metrics.

Here, for example, is the number of requests made in this application:

Interesting, right?

2. Collection and storage using prometheus

So the next step now is to collect and store these metrics somewhere. For this we will use Prometheus.

In the documentation we have the metrics systems that are supported. One of them we will use is Prometheus. To use Prometheus, it is necessary to include the Micrometer library. To support Prometheus.

https://prometheus.io/docs/prometheus/latest/getting_started/

https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.metrics.export.prometheus

2.1 Add micrometer library

Micrometer is a Java library that works as a facade for collecting and exporting metrics. So it supports Promethues and many other metrics collection and export systems.

Here is the microemeter documentation:

https://docs.micrometer.io/micrometer/reference/implementations/prometheus

We include micrometer dependency with latest pinned version.

<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
  <version>${micrometer.version}</version>
</dependency>

Let’s configure it to expose only the metrics we want in the properties.

http://localhost:8080/actuator/prometheus

The format that prometheus accepts being exposed:

With the Prometheus endpoints created exposing the metrics. Let’s configure and upload a promehues service using Docker.

To do this, we will need to have Docker installed on our machine. We will need to create a docker-compose file, Dockerfile, and a prometheus.yml within our project.

docker-compose.yml

version: '3.3'

networks:
  aw-networks:
    driver: bridge
    
volumes: 
  prometheus_data:
  
services:
  prometheus:
    image: prom/prometheus:latest
    volumes:
     - ./config/prometheus.yml:/etc/prometheus/promethues.yml
     - prometheus_data:/prometheus
    networks:
     - aw-networks
    ports:
     - 9090:9090

Dockerfile

# Use a Java base image
FROM openjdk:11-jre-slim

# Copy your application's JAR file to the container
COPY target/<your-service>-0.0.1-SNAPSHOT.jar /app/<your-service>-0.0.1-SNAPSHOT.jar

# Exposes the application port
EXPOSE 8080

# Command to start the application when the container is started
CMD ["java", "-jar", "/app/<your-service>-0.0.1-SNAPSHOT.jar"]

/config/prometheus.yml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'your-service'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['host.docker.internal:8080']
        labels:
          application: "<your-service>"

With these configurations prepared, initialize the docker container using docker-compose

docker-compose up

And when accessing the service configured on port 9090. We will have the Prometheus graphical interface:

Accessing Prometheus at:

http://localhost:9090/graph?g0.expr=&g0.tab=1&g0.display_mode=lines&g0.show_exemplars=0&g0.range_input=1h

But when accessing the Prometheus service run by the application, we noticed that communication did not occur. as this occurs due to the Prometheus application and service being on different hosts. Where one runs on localhost and the other runs inside a container.

To solve this and make Prometheus see the applications running on the host, we will pass the Docker DNS into the Prometheus property.

- targets: ['host.docker.internal:8080']

Then Prometheus starts to see the API.

Prometheus already has some dashboards in its features. As observed;

But we will use another tool that is more robust for dashboards, which will be Grafana. Let’s go?

3. Using Grafana

Grafana has observability properties but it is also a system that offers several dashboards that integrate with Prometheus.

I will configure grafana in Docker but it is also possible to configure grafana on your local machine. For this, I suggest looking at the official grafana documentation.

Let’s include the grafana settings in docker to generate an image of it:

version: '3.3'

networks:
  aw-networks:
    driver: bridge
    
volumes: 
  prometheus_data:
  grafana_data:
  
services:
  prometheus:
    image: prom/prometheus:latest
    volumes:
     - ./config/prometheus.yml:/etc/prometheus/prometheus.yml
     - prometheus_data:/prometheus
    networks:
     - aw-networks
    ports:
     - 9090:9090
  grafana:
    image: grafana/grafana:latest
    ports: 
     - 3000:3000
    networks:
     - aw-networks

To the container and climb back up.

docker-compose up

An image of grafana will be created.

Running at address:

http://localhost:3000/login

When logging in, just type admin admin.

The first step is to configure a Datasource. This will be where grafana will query the data.

Include the promethues dns.

So in exploring, we will be able to see the metrics exposed by Prometheus.

Select a metric and Run the Query. We will have a dashboard.

By default, grafana displays the dashboard in its own template. However, we can obtain several dashboard templates on the Grafana website. These dashboards were developed by people or companies and are available for use.

Then we access the grafana website at https://grafana.com/ and at the end of the page we select dashboards https://grafana.com/grafana/dashboards/;

We will have several dashboard templates for different tools and needs.

We will use the Micrometer dashboard. Where we will type in micrometer search. https://grafana.com/grafana/dashboards/4701-jvm-micrometer/

And to use it, we copy the dashboard id. Then go to grafana in Dashboards, create a new one, select the import option and paste the ID.

Select the Source for Promethues and we will have the dashboards already created.

There are dashboards for Spring. This micrometer dashboard only has metrics from the JVM itself, it does not have metrics based on requests. Then we will import another Sping dashboard template. Go back to the Grafana website and in dashboards search for spring. Several dashboards will be displayed and we will use “Spring Boot Statistics & Enpoints metrics”

Let’s copy the id again and then import a new dashboard:

We will need to do a little configuration. Go to dashboard settings.
In variables, leave only application:

label_values($application)

When doing this, note that the prev of values will display the name of the service label:

Make the same configuration for the instance variable

label_values(jvm_classes_loaded_classes{application="$application"},instance)

Again in prev of values the value below should be:

Finally, with these settings, we can go to the spring dashboard and analyze the requests.

So, accessing my documentation, I make some requests:

I go to the Grafana requests dashboard and there are the updated metrics:

And we have reached the end of another publication, I hope this article is useful to someone in need of help.

References:

• [Monitoring system with Spring prometheus] https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.metrics.export.prometheus
• [Prometheus documentation] https://prometheus.io/docs/prometheus/latest/getting_started/
• [Grafana documentation and templates] https://grafana.com/
• [Micrometer — promethes] https://docs.micrometer.io/micrometer/reference/implementations/prometheus