Producing Kafka Messages in Quarkus

Lately, in our team in Trendyol, in this particular API, we wanted to send messages to a topic, and we were pretty used to doing it in Spring, but it was our first try in Quarkus.

We’ve spent lots of time getting the hang of it. So, I wanted to share how you can implement it and to save you the trouble.

Eclipse Microprofile Reactive Messaging (the Reactive Messaging Client which Quarkus’ official documentation uses) did not support what we wanted to achieve.

It provides an @Outgoing annotation that can be triggered by an incoming message to the API. In our case, there was not going to be an incoming message, and we wanted our API just to produce a message. Simple as that.

To achieve this, we used Vert.x’s Kafka Client.

Let’s start:

If you use Maven, add the following dependency to your pom.xml file

or if you use Gradle, add the following dependency to your build.gradle

Since Vert.x’s Kafka Client doesn’t automatically read values from application.properties (or .yml/.yaml) file, we need to implement it ourselves, just like a regular configuration class.

application.yml:

If you plan to send messages to different topics with different themes in your program, the best practice would be to make bootstrap-servers, key-serializer, and value-serializer common (since they won’t change), and make topic more abstract, so you can add different topics to use same config values, such as:

So these values need to be read by Java. :)

Just a usual configuration class, nothing special here.

Now let’s implement our KafkaProducer service:

Okay, we’ve made our injections.

Now we need to actually put our config values into our kafkaProducer.

The @PostConstruct annotation here makes sure this code is run after initialization.
So now, kafkaProducer has been initialized with our config values.

Thus, we created a record with our topic and message, then sent it using kafkaProducer, which had our server information.

Since it is asynchronous, we need to return a Future type.

In this particular example, I am not interested in its return value, so I am just returning Future<Void> with null inside. You can choose your return type according to your own needs.

kafkaProducer itself is asynchronous, indicating it will start sending when kafkaProducer.write(record); is called and won’t care if it’s finished or not and the program will continue running.

BUT to make the whole process asynchronous (if you want to call the method from another method/class and continue running), you need to put @Asynchronous annotation as seen above.

Again, you can choose how to implement it according to your own needs.

Let’s test it:

I am connecting to our topic from a local Kafka server and sending a message using our program.

Voila!

The reason why I wrote this story was that I found plenty of simple stories & articles about Kafka in Spring, but not so much in Quarkus.

I hope it was helpful. :)

Quarkus is a fairly new and fast-developing technology. Please keep in mind that this story was written on Sep 2020 and may get deprecated in time.

Thank you for reading! ❤️

Thanks to all my colleagues in the Browsing Team. 🤟