Building your own OpenTelemetry Collector distribution

It doesn’t always work on the first try…

The OpenTelemetry Collector is designed to be extensible, allowing custom extensions, receivers, processors, and exporters to be plugged into the pipeline. The problem is that Go’s support for plugins is still far away from prime-time. And instead of bloating the core distribution with components that might not be useful to everyone, the opentelemetry-collector-contrib was created: it’s a repository where each component is a Go module, meaning that they are meant to be consumed individually. To make it easier for people who want to use an existing component from the contrib repository, an “otelcontribcol” binary exists for each release.

But what if you don’t want your collector to include all of the growing list of components in contrib, and instead just want to be selective about the components you want to include? And how about adding your own components?

For those cases, you have to build your own OpenTelemetry Collector distribution. But it’s not as hard as it sounds, as this guide will show you.

Step 1: Ready to Go

The first step is to get the right tools. For this project, all we need is Go. You should follow the official Go installation guide for your platform, but if you are a Fedora user, you can just use dnf:

$ dnf install golang

Step 2: Creating a new project

The next step is creating the scaffolding for the new project:

$ cd ~/Projects/otelcol-custom
$ go mod init github.com/jpkrohling/otelcol-custom

Change github.com/jpkrohling to your own namespace, possibly using a meaningful name for your distribution.

Every Go program starts with a main.go, containing a main function. With the following main.go , we have our own OpenTelemetry Collector distribution, with all the components available in the core distribution:

main.go

The first step is to get a list of components that we want in our distribution. We do that by calling defaultcomponents.Components().

The second step is to build our ApplicationStartInfo, containing the name of our application, a description and a version.

We then create a “service”, which is the actual OpenTelemetry Collector, using the two pieces of information that we have already (components, application start info).

And finally, we start the collector!

Step 3: Testing our distribution

Time to do a sanity check: are we able to compile our distribution? Can we actually run it?

All we have to do is run go run . --config config.yaml , where the config.yaml may look like this:

config.yaml

Running it for the first time should take a while, as Go will fetch all the dependencies and compile a temporary binary, but once it’s all ready, your OpenTelemetry Collector should start fine, blocking with a line containing “Everything is ready. Begin running and processing data.”

Step 4: Building our list of components

At this stage, all we have is a distribution that is functionally identical to the core OpenTelemetry Collector. We can increment it by adding our own components. All we have to do is to replace factories in the line #11 from our main.go with a list that we are building ourselves.

main.go calling our “components()”

In a separate source file named components.go, we build a components() function that takes care of building the list of desired components:

components.go, but still without custom processors

This is the main scaffolding we need. From this point and on, adding processors is easy: just add them to the slice between the lines #16 and #17. But first, time for a sanity check: go run . --config config.yaml.

Step 5: Adding our custom processor

The only thing missing within our source code is to actually add a custom processor. For this exercise, we are adding Observatorium’s Authentication processor:

components.go with the authentication processor

We can now reference our new processor in the config, which is sufficient to confirm that it’s available to our distribution:

config.yaml with the authentication processor

For simplicity, we left out possible extra extensions, receivers or exporters that we might want, but the process is very similar to what we are doing with the processors: get a list of factories for the component type, include the default components for the component type, make a new component map and override the map of factories in the factory list (lines #19 to #28 in the components.go above).

Step 6: It’s compiling!

Now that we have everything in place, it’s time to compile and build our binary:

$ go build -o otelcol-custom .

At the end of this process, you’ll have a new binary named otelcol-custom in the current directory. We can just run it similar to how we used go run before:

$ ./otelcol-custom --config config.yaml

The difference is that it now starts up almost instantaneously.

Bonus chapter

You probably realized that assembling an OpenTelemetry Collector distribution is mostly a mechanical process, tying things from different sources together. If you are not a Go developer, or if you prefer to have this automated in some way, you can try the OpenTelemetry Collector Builder utility, where you can specify the components you want in a YAML file and let the builder assemble an OpenTelemetry Collector distribution for you.

Summary

Building a custom OpenTelemetry Collector distribution isn’t really complicated, especially for seasoned Go developers: just create a main function tying all the pieces together and compile this code.

For non-Go developers, tools such as the OpenTelemetry Collector Builder can help, requiring only a manifest file to be provided.