The snowplow-java-tracker in a multi-client service architecture
And one library to rule them all
In this blog post, we will talk about the open-source library written in Kotlin which we created in order to simplify the implementation of the snowplow-java-tracker in our projects. Before we dive deeper into it let’s understand how Snowplow is used for events collection in our organization throughout our projects.
At upday we utilize a Snowplow pipeline approach to tracking user-events and store them in our Data Warehouse. We track events in web-based, native frontend, and backend applications. This is visualized in the below diagram.
The problem and our solution
We use the snowplow-java-tracker in multiple microservices. In order to do so, we had to implement the tracker component and its configuration as well as the emitter again in each of the services.
Also, any bug or change in the tracker or emitter needed a patch in all the services that use the snowplow-java-tracker. Additionally, to make our system more robust we had to attempt retrials for failed events in case of downtime of the event collector endpoint.
We abstracted the tracker and emitter creation to a separate library snowplow-java-tracker-wrapper with default tracker and emitter properties. This library can now be added as a dependency in our services which implements the snowplow-java-tracker. We also enhanced our library to take care of failed event retrials in case the snowplow collector URL is down with up to five exponentially backed-off retrials by default.
How to use the Library
In order to get started, add the JitPack repository together with the snowplow-java-tracker-wrapper dependency to your project. There you can find the latest released version of the library.
To start sending your events, you need to instantiate the SnowplowDispatcher class with your application properties. This takes care of creating the tracker and the emitter. After that, you need to call the send method that submits your events to the collectorUrl.
But what about the performance?
We have been using this library to collect events in our production environment for the last four months.
So far the performance has been stable where we have a traffic of about 100 requests/sec, generating snowplow events at a similar rate. Here is a performance report of one of our services using this library to send events.
However, we recommend you to run a load test based on your expected traffic before using this library.
Happy event sending :)
We hope you enjoyed reading our blog post and we would be happy if someone utilizes or even improves the library. Please feel free to contribute to our open-source library snowplow-java-tracker-wrapper. We thank all our contributors so far.
Cheers, Monika Singh and Julian Dobrot :)
