kafkacat on AmazonLinux2 / CentOS

Image for post
Image for post
Photo by Bing Han

When I prepared some demos about Amazon Managed Streaming for Kafka (Amazon MSK) I realised that one of the CLI-tools that I like to use was not available on my EC2 instance: the useful open-source kafkacat tool.

There seems to be no binary available for Amazon Linux via yum install, but you can quickly build it yourself. Doing so requires UNIX make, as well as a C and C++ compiler (so the whole task mentally throws you back to the time before the “write once, run anywhere” became a thing).

kafkacat Installation on Amazon Linux2

To build kafkacat yourself, just run the following commands on the EC2 instance that you want to use as a client (where you want to install kafkacat):

You might get away with less then the “Development Tools” group install, let me know if you find out.

Examples

Then try out your newly installed kafkacat tool, e.g. by using one of the following commands:

You can use the AWS MSK console to retrieve the bootstrap servers easily:

For an MSK cluster set up with plain text, you can query for the brokers as shown below:

In this example I have 6 broker running across 3 availability zones:

More examples

There are many other examples that you might find useful when running Apache Kafka especially on AWS. I will continue this posting with some more examples… stay tuned!

This article about calling MSK via REST might be interesting.

Please clap for this article if you enjoyed reading it as much as I enjoyed writing it. I spend way too much time on Twitter, so feel free to connect: @frankmunz.

Written by

Technical Evangelist @AWS, twitter @frankmunz. My personal opinions here :-)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store