Distributed Tracing with Spring Boot and Jaeger

Published in

Javarevisited

4 min readNov 8, 2021

Distributed Tracing with Spring Boot and Jaeger

Introduction

Distributed tracing provides the insight into the flow and lifecycle of a request as it passes through a system. Modern day platforms may be split across many different isolated services, all of which may contribute to produce a final result. In a microservices style architecture a single client request could spawn a number of subsequent requests into various different areas components, which in turn may perform additional downstream requests. In addition, this might not be over the same protocol — HTTP via RESTful endpoints, perhaps various types of queues etc. As the logs for each of these components are separated, it can be extremely difficult and time consuming to track the series of events as it flows through different areas.

We have understood the need for Distributed Tracing in Microservices. but we need some tools to achieve that. Jaeger is a distributed tracing platform — originally developed by Uber. It is used for monitoring and troubleshooting issues with Microservices based architecture. It has a nice UI which clearly shows the complete request details & processing times etc.

Technologies used in the article:

Spring boot version: 2.5.6
jaeger
Java version 1.8

Create a spring boot application

Create a spring boot application with the required dependencies. Add the spring boot starter web dependency. Also, include jaeger dependencies to the spring boot application’s pom XML configuration file.

The below is the complete content of the pom.xml file.

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <parent>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-parent</artifactId>
  <version>2.5.6</version>
  <relativePath /> <!-- lookup parent from repository -->
 </parent>
 <groupId>com.example.demo.jaeger</groupId>
 <artifactId>spring-boot-distributed-trace-jaeger</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <name>spring-boot-distributed-trace-jaeger</name>
 <description>Demo project for Spring Boot</description>
 <properties>
  <java.version>1.8</java.version>
 </properties>
 <dependencies>
  <dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter</artifactId>
  </dependency>
  <dependency>
   <groupId>io.opentracing.contrib</groupId>
   <artifactId>opentracing-spring-jaeger-cloud-starter</artifactId>
   <version>3.3.1</version>
  </dependency>
  <dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-test</artifactId>
   <scope>test</scope>
  </dependency>
 </dependencies><build>
  <plugins>
   <plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
   </plugin>
  </plugins>
 </build></project>

Adding Controller class

Create a REST API class with the name Controller and add the below content.

@RestController
@RequestMapping("/hello")
public class Controller {private static final Logger logger = LoggerFactory.getLogger(Controller.class);
    
private RestTemplate restTemplate;@Value("${spring.application.name}")
    private String applicationName;public Controller(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }@GetMapping("/path1")
    public ResponseEntity<String>  path1() {logger.info("Incoming request at {} for request /path1 ", applicationName);
        String response = restTemplate.getForObject("http://localhost:8090/hello/path2", String.class);
        return ResponseEntity.ok("response from /path1 + " + response);
    }@GetMapping("/path2")
    public ResponseEntity<String>  path2() {
        logger.info("Incoming request at {} at /path2", applicationName);
        return ResponseEntity.ok("response from /path2 ");
    }
}

The controller exposes a GET API with the endpoint /hello/path1 and returns the calculated path1 response. Similarly with another GET API with the endpoint /hello/path2 and returns the calculated path2 response.

Here, we have two endpoints /path1 and /path2 . The idea here is to use two instances of the same application such that/path1 calls /path2 of another service at a fixed port 8090.

For the spans to get connected to the same trace id, We need to create a RestTemplate bean to allow Jaeger to include an interceptor. This then helps to add traces to the outgoing request which will help to trace the entire request.

@Bean
   public RestTemplate restTemplate(RestTemplateBuilder builder) {
      return builder.build();
   }

Docker Compose

Let’s start a Jaeger Server locally using docker. For this, I have created a docker-compose file with the port mappings.

version: "3.3"
services:
  jaeger-allinone:
    image: jaegertracing/all-in-one:1.25
    ports:
      - 6831:6831/udp
      - 6832:6832/udp
      - 16686:16686
      - 14268:14268

We can communicate with Jaeger using either via UDP or TCP. After starting the docker image using docker-compose up , we can access the UI using the URL http://localhost:16686/

Now, let’s add some properties to allow the application to send the traces to the Jaeger server. We will communicate via TCP, so make sure that we send the traces to the other TCP port. i.e 14268

opentracing:
  jaeger:
    http-sender:
      url: http://localhost:14268/api/traces

Starting Service 1

Let’s start “Server 1” with the below command.

java -jar target/spring-boot-distributed-trace-jaeger-0.0.1-SNAPSHOT.jar --spring.application.name=Service-1 --server.port=8080

Starting Service 2

On a different terminal, run a new instance of the same application as “Service 2” as follows

java -jar target/spring-boot-distributed-trace-jaeger-0.0.1-SNAPSHOT.jar --spring.application.name=Service-2 --server.port=8090

Once the application starts, call “Service 1” at /path1 as follows

curl -i http://localhost:8080/hello/path1

Let’s look at the logs of “Service 1”.

INFO 16636 --- [nio-8080-exec-2] i.j.internal.reporters.LoggingReporter   : Span reported: b7c9ce2661df9d7c:c3b96d4fbeb65d4a:b7c9ce2661df9d7c:1 - GET

The tracing is of the format [Root Span Id, Current Span Id, Parent Span Id]. In this case, since “Service 1” is the originating service, the parent span Id “b7c9ce2661df9d7c” is also the root span id.

Now, let’s look at the logs of “Service 2”.

INFO 22732 --- [nio-8090-exec-1] i.j.internal.reporters.LoggingReporter   : Span reported: b7c9ce2661df9d7c:bf420c75d1105309:c3b96d4fbeb65d4a:1 - path2

Here we see that the middle value is the current span id and the parent span id (ie. the third value “c3b96d4fbeb65d4a”) is the span id of “Service 1”.