Running Jaeger Agent on bare metal

The Jaeger Agent is the component that is responsible for receiving the spans from the instrumented application and forwarding it to the Jaeger Collector, so that the data gets appropriately stored. In addition to acting as a span buffer between the application and the collector, the Jaeger Agent also receives updates from the collector regarding the sampling strategies, making said strategies available via a REST endpoint that is queried by the Jaeger Client, deployed within the instrumented application.

The Jaeger Client typically sends spans via UDP to the agent, avoiding the TCP overhead and reducing the CPU and memory pressure upon the instrumented application. With that in mind, the Jaeger Agent should be deployed as close as possible to the instrumented application, reducing the risks inherent to UDP delivery. A second aspect to take into consideration is the tenancy model, as each Jaeger Agent is typically used by a single tenant.

With the single-tenant scenarios, Jaeger Agent instances can be shared among multiple instrumented applications. This means that the agent runs as a daemon process in the same machine as the applications.

With the multi-tenant scenarios, there should be one Jaeger Agent per tenant, meaning that there are multiple agent daemon processes running, one per tenant.

The following SystemDservice unit file can be used to control the Jaeger Agent lifecycle. The example uses a YAMLconfiguration file located at /etc/jaeger-agent.yamland assumes that the Jaeger Agent binary is located at /usr/local/bin/agent-linux.

With that in place, the Jaeger Agent can be started by issuing the following command:

The status and logs can be queried like any other process managed by SystemD, like:

An empty configuration file at /etc/jaeger-agent.yamlis sufficient to enable an Agent to properly start with a collector running on localhost. On production environments, it’s recommended to have the collectors running as a cluster in their own hosts. A typical configuration file would therefore include at least the location of the Jaeger Collector, such as:

In a multi-tenant scenario, the “instantiated service” feature of SystemDis suitable. In that case, the unit file would look like:

Given a configuration file located at /etc/jaeger-agent-tenant1.yaml, start the agent for the tenant tenant1with:

As a good practice, explicitly set the host-port properties the agent should bind, like the following:

The second tenant should then bind to different ports, like:

Starting the second tenant with systemctl start jaeger-agent@tenant2, there should be two Jaeger Agent instances running, each on its own set of ports. The example above also uses a different hostname for the target collector, used exclusively to handle spans from our second tenant.

We’ve seen here how we can make use of SystemD to manage our Jaeger Agent process, both with a single and multi-tenant scenario. With some simple changes, the same strategy can be used to manage Jaeger Collector and Jaeger Query processes.

How to deploy in Kubernetes and OpenShift is a topic for a future blog post. In the meantime, take a look at the Jaeger Operator, which handles most of the process for you.