Generating Jaeger gRPC services and using Jaeger in JUnit with Testcontainers

Published in

JaegerTracing

4 min readApr 8, 2020

In this article you will learn how to generate Jaeger model classes and gRPC services from protobuf definitions. The generated code can be used to write integrations or it can be used in tests to query tracing data from Jaeger-query service. This is the use case we are going to look at in more detail.

Generating Java classes from proto definitions

Anyone working with protobuf files agrees that it is kind of a nightmare to get it right, with having to install the right dependency versions of all packages. To ease this process of generating gRPC services from proto definitions, we have created a lightweight docker image (47MB), with the protoc compiler and package dependencies baked in. The Dockerfile is hosted at jaegertracing/docker-protobuf, the image at JaegerTracing DockerHub, and is synced with the dependencies from the Jaeger core repo. It can be used to generate services in multiple languages including GoLang, Java, Python, PHP, etc.

To generate model and gRPC service classes in Java use the following commands:

git clone git@github.com:jaegertracing/jaeger-idl.gitcd jaeger-idl && mkdir java_outdocker run --rm -it -v${PWD}:${PWD} -w${PWD} \
    jaegertracing/protobuf:latest --proto_path=${PWD} \
    -I/usr/include/github.com/gogo/protobuf \
    --java_out=${PWD}/java_out proto/api_v2/model.proto

The flag java_out is used to specify that we want the generated code for Java. For a complete list of options, run:

docker run --rm -it jaegertracing/protobuf:latest --help

Sometimes, the generated code in the specific language might have dependencies on packages like GoGo or Swagger, as these are often used to make the internal representations more efficient. We expect that proto definition files for all dependencies are present in the image. The same image can be used to generate code from the proto definition file of the dependency. To find all proto files present in the image, run:

docker run --rm -it --entrypoint=/bin/sh jaegertracing/protobuf:latest -c "find /usr/include -name *.proto"

And once the dependency file is located, language specific code can be generated using the same command above.

Please note that detailed guidelines for code generation are available in the README.md in the Dockerfile repo.

Maven artifacts

Generated Java classes are also available as Maven artifacts, so they can be imported into the project directly as follows:

<dependency>
  <groupId>io.jaegertracing</groupId>
  <artifactId>jaeger-proto</artifactId>
  <version>${version}</version>
</dependency>

Using Jaeger in tests

There might be many use cases for using Jaeger in tests. The most common one is to verify whether a service is reporting tracing data or even a better use case is to use tracing in end-to-end tests to validate the system behavior by looking at reported trace data. Btw. you might want to check out Trace Driven Development: Unifying Testing and Observability talk by Ted Young where he is talking about using tracing/telemetry data in tests.

In this example we will have a look at using Jaeger in Java. The all-in-one distribution will be directly deployed in the test by Testcontainers library. This library makes it really simple to use docker services in JUnit tests.

Let’s have a look at how Jaeger all-in-one deployment can be implemented as GenericContainer:

public class JaegerAllInOne extends GenericContainer<JaegerAllInOne> {

  public static final int JAEGER_QUERY_PORT = 16686;
  public static final int JAEGER_COLLECTOR_THRIFT_PORT = 14268;
  public static final int JAEGER_COLLECTOR_GRPC_PORT = 14250;
  public static final int JAEGER_ADMIN_PORT = 14269;
  public static final int ZIPKIN_PORT = 9411;

  public JaegerAllInOne(String dockerImageName) {
    super(dockerImageName);
    init();
  }

  protected void init() {
    waitingFor(new BoundPortHttpWaitStrategy(JAEGER_ADMIN_PORT));
    withEnv("COLLECTOR_ZIPKIN_HTTP_PORT",    
         String.valueOf(ZIPKIN_PORT));
    withExposedPorts(JAEGER_ADMIN_PORT, 
         JAEGER_COLLECTOR_THRIFT_PORT, 
         JAEGER_COLLECTOR_GRPC_PORT,
         JAEGER_QUERY_PORT,
         ZIPKIN_PORT);
  }

  public int getCollectorThriftPort() {
    return getMappedPort(JAEGER_COLLECTOR_THRIFT_PORT);
  }

  public int getQueryPort() {
    return getMappedPort(JAEGER_QUERY_PORT);
  }

  public JaegerTracer createTracer(String serviceName) {
    String endpoint = 
        String.format("http://localhost:%d/api/traces", 
        getCollectorThriftPort());
    Sender sender = new HttpSender.Builder(endpoint)
        .build();
    Reporter reporter = new RemoteReporter.Builder()
        .withSender(sender)
        .build();
    Builder tracerBuilder = new Builder(serviceName)
        .withSampler(new ConstSampler(true))
        .withReporter(reporter);
    return tracerBuilder.build();
  }

  public QueryServiceBlockingStub createBlockingQueryService() {
    ManagedChannel channel = 
       ManagedChannelBuilder.forTarget(
           String.format("localhost:%d",   
           getQueryPort())).usePlaintext().build();
    return QueryServiceGrpc.newBlockingStub(channel);
  }

  public static class BoundPortHttpWaitStrategy extends HttpWaitStrategy {
    private final int port;

    public BoundPortHttpWaitStrategy(int port) {
      this.port = port;
    }

    @Override
    protected Set<Integer> getLivenessCheckPorts() {
      int mapptedPort = this.waitStrategyTarget.getMappedPort(port);
      return Collections.singleton(mapptedPort);
    }
  }

There are a couple of interesting things to point out:

method createTracer returns a configured tracer instance that reports data to the deployed all-in-one instance.
method createBlockingQueryService returns gRPC query service.
the deployed all-in-one exposes Jaeger collector and query ports. Note that these ports are mapped to a random host port. The mapped port can be obtained via JaegerAllInOne.getMappedPort method.

Now let’s have a look at how to use this class in the test:

public class TracingTest {  private static final String SERVICE_NAME = "query-test";  private JaegerAllInOne jaeger = new 
     JaegerAllInOne("jaegertracing/all-in-one:latest");
  private Tracer tracer;

  @Before
  public void before() {
    jaeger.start();
    tracer = jaeger.createTracer(SERVICE_NAME);
  }

  @After
  public void after() {
    jaeger.stop();
    tracer.close();
  }

  @Test
  public void testGetService() {
    tracer.buildSpan("test-operation")
        .withTag("foo", "bar")
        .start()
        .finish();

    QueryServiceBlockingStub queryService = 
        jaeger.createBlockingQueryService();
    WaitUtils.untilQueryHasTag(queryService,
        SERVICE_NAME, "foo", "bar");

    GetOperationsResponse operations = queryService
     .getOperations(GetOperationsRequest.newBuilder()
          .setService(SERVICE_NAME).build());
    Assert.assertEquals(1, 
        operations.getOperationNamesList().size());
    Assert.assertEquals("test-operation", 
        operations.getOperationNamesList().get(0));
  }
}

In this case a Jaeger all-in-one instance is created per each test run. This is because Jaeger does not provide an API to remove old data which might be necessary to do between test runs. The alternative approach without requiring to restart the server might be to add a unique tag to each span in a test and include it in the query.

Jaeger Java artifact with Testcontainers

The JaegerAllInOne class from the previous section is also available as a Maven artifact. The source code is hosted in the jaegertracing/jaeger-analytics-java repository.

<dependency>
  <groupId>io.jaegertracing</groupId>
  <artifactId>jaeger-testcontainers</artifactId>
  <version>${version}</version>
</dependency>

References

Jaeger docker-protobuf repository: https://github.com/jaegertracing/docker-protobuf
Jaeger analytics-java repository: https://github.com/jaegertracing/jaeger-analytics-java
Testcontainers https://www.testcontainers.org/
Protobuf documentation: https://developers.google.com/protocol-buffers/docs/proto3

This article has been written by Annanay Agarwal and Pavol Loffay.