Building Spring Boot Microservices , Monitoring with prometheus and grafana and log aggregation using ELK stack: Part II

In the first part we learned how to build spring boot microservices by creating 3 microservices (Product-Microservice, Order-Microservice, Client-Microservice).

As for now everything looks fine, but while working with such a complex architecture we need to keep an eye on our system in case of a any potential failure.

To do so , in this part we’re gonna see how to monitor,trace and observe the behaviour of our microservices using prometheus and grafana.

What is prometheus ?

Prometheus is an open source time-series database originally developed by SoundCloud. It comes with its proper query language PromQL and used for event monitoring and alerting.

Adding Micrometer Prometheus Registry to Spring Boot microservice

Spring boot uses Micrometer an application metrics facade to integrate actuator metrics with external monitoring systems.

It supports several monitoring systems like SignalFx, Graphite, Wavefront, Prometheus …

To integrate actuator with Prometheus, you need to add the following dependencies to each of your microservices:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

and the following properties:

management.endpoint.metrics.enabled=true
management.endpoints.web.exposure.include=*
management.endpoint.prometheus.enabled=true
management.metrics.export.prometheus.enabled=true

Once you add the above dependency, Spring Boot will automatically configure a PrometheusMeterRegistry to collect and export metrics data in a format that can be scraped by a Prometheus server.

To explore prometheus endpoint go to http://localhost:9001/actuator

we’re gonna work on the product micoroservice for example.

actuator

as we can see , actuator exposes a lot of endpoints , we’re just gonna focus on prometheus endpoint.

prometheus endpoint

Downloading and installing prometheus

Prometheus is available as a docker image and if you don’t have docker you can download it from here https://prometheus.io/download/.

we’re not gonna work with docker because not everyone is familiar with it.

After installing prometheus(with or without docker) , we have to create prometheus.yml file to configure Prometheus to scrape metrics data from Spring Boot Actuator’s /prometheus endpoint.

The configuration should look like this :

scrape_configs:
  - job_name: 'microservice-product'
    scrape_interval: 2s
    metrics_path: '/actuator/prometheus' ##prometheus endpoint
    static_configs:
      - targets: ['localhost:9001'] ## host and port for your mis
  - job_name: 'microservice-order'
    scrape_interval: 2s
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['localhost:9002']
  - job_name: 'microservice-client'
    scrape_interval: 2s
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['localhost:8080']

Running prometheus

Open your terminal on your prometheus folder and run the following command:>

prometheus.exe --config.file <PATH TO YOUR PROMETHEUS.YML>

Navigate to http://localhost:9090 and explore prometheus.

Once all our microservices are up we should have something like that:

targets

We can now explore all the metrics in prometheus and keep an eye on the behaviour of our microservices , this example shows the system’s CPU usage of our microservices:

Grafana

So far our microservices are exposing the actautor’s endpoint into prometheus server which is so interesting , but when we have this amount of information we need another system to help us better visualize , analyse our microservices.

Grafana is open source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (prometheus for example )data into beautiful graphs and visualizations.

Downloading and installing grafana

You can download grafana from here https://grafana.com/grafana/download then go to bin under grafana and execute the grafana-server.exe

Grafana run by default on port 3000, navigate to http://localhost:3000 and log in to Grafana with the default username admin and password admin.

grafana

Configuring Grafana to import metrics data from Prometheus

Like we said before grafana imports metrics from time series database in order to visualize them , to do so we need to configure grafana to import these metrcis from the prometheus server.

the following image shows how grafana and prometheus interact with each other:

architecture

To import prometheus metrcis into grafana we have to follow these steps:

1. Add the Prometheus data source in Grafana

go to configuration/datasource and click to add new datasource

Grafana provides a lot of databases , we just need to select prometheus

After selecting your TSDB , add the following configuration to connect garafana with your prometheus server :

2.Create a new dashboard

Select the (+) icon form the left menu and click dashboard:

3.Visualize your dashboard

Importing pre-built dashboards

Grafana.com maintains a collection of shared dashboards which can be downloaded and used with standalone instances of Grafana. Use the Grafana.com “Filter” option to browse dashboards for the “Prometheus” data source only.

we’re gonna search for the spring boot statistics dashboard for example:

copy the id and go to dashboard/import and click load

Your dashboard is ready to explore. Now it’s up to you andr needs , you can do whatever you want and you can keep and eye on your whole system.
In the next and final part we’re gonna see how to aggregate the microservices logs with Elastic stack.

You can find the update source code on the git repository:

FirasMessaoudi/ecommerce

I’ll notify you in the comments when the final part is ready.