kafkacat on AmazonLinux2 / CentOS

Frank Munz
Jun 14 · 2 min read
Photo by Bing Han

When I was preparing some demos about Amazon Managed Streaming for Kafka (Amazon MSK) I realised that one of the CLI-tools that I like to use was not available on my EC2 instance: the useful open-source kafkacat tool.

There seems to be no binary available for Amazon Linux via yum install, but you can quickly build it yourself. Doing so requires UNIX make, as well as a C and C++ compiler (so the whole task mentally throws you back to the time before the “write once, run anywhere” became a thing).

kafkacat Installation on Amazon Linux2

To build kafkacat yourself, just run the following commands on the EC2 instance that you want to use as a client (where you want to install kafkacat):

sudo yum update -y
sudo yum install git -y
# make sure you have the necessary dev tools avail,
# or run the ff command:
sudo yum group install “Development Tools” -y
git clone https://github.com/edenhill/kafkacat.git
cd kafkacat/
./bootstrap.sh
./kafkacat -h

You might not need or get away with less then the “Development Tools” group install, let me know if you find out.

Examples

Then try out your newly installed kafkacat tool, e.g. by using one of the following commands:

./kafkacat -h./kafkacat -L -b YOUR.kafkabroker.com:9092

You can use the AWS MSK console to retrieve the bootstrap servers easily:

For an MSK cluster set up with plain text, you can query for the brokers as shown below:

./kafkacat -L -b b-1.kafkathree.k4hgaf.c3.kafka.eu-central-1.amazonaws.com:9092,b-3.kafkathree.k4hgaf.c3.kafka.eu-central-1.amazonaws.com:9092,b-6.kafkathree.k4hgaf.c3.kafka.eu-central-1.amazonaws.com:9092

In this example I have 6 broker running across 3 availability zones:

More examples

There are many other examples that you might find useful when running Apache Kafka especially on AWS. I will continue this posting with some more examples… stay tuned!

This article about calling MSK via REST might be interesting.

Frank Munz

Written by

Technical Evangelist at AWS

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade