Apache Kafka Guide #18 Changing Topic Configuration CLI

Hi, this is Paul, and welcome to the #18 part of my Apache Kafka guide. Today we will discuss how to change Apache Kafka Topic Configuration.

Apache Kafka At the Gates of Mastery: Kafka Core

Adjusting the settings for topic configurations

But why is this important?

Brokers come with default settings for all topic configuration options. These settings affect both performance and how your topics behave. Therefore, for certain topics, you might need to change these settings from the default broker values.

This could involve altering the replication factor, number of partitions, message size, compression level, log cleanup policy, minimum in-sync replicas, and other settings.

You can find a complete list of Kafka configurations for topics at the provided URL.

Specifically, we’ll adjust the “Min Insync Replicas” setting to two for one of our topics. To begin, we need to create a topic. We’ll start by initiating a create command. During this process, we will specify the bootstrap server as localhost:9092. Our focus will be on a particular topic, which I will name ‘configure topic’. This topic will be set with a replication factor of one and assigned three partitions.

kafka-topics --create --bootstrap-server localhost:9092 --topic my-topic --replication-factor 1 --partition 3

The topic is created.

Next, I need to verify if the topic already has an existing configuration. I’ll use the same command but omit ‘create’, ‘replication factor’, and ‘number of partitions’. Instead, I’ll add ‘ — describe’ to get details about the topic.

kafka-topics --bootstrap-server localhost:9092 --topic my-topic --describe

This will show the topic’s name, ID, partition count, replication factor, and configurations, which are currently empty. We’ll then add a configuration to our topic and observe the outcome.

So for this,

we have a command called the Kafka-Configs command.

kafka-config

You can access the complete documentation here for that configuration. Kafka-Configs enables you to add a configuration. For topics as an entity type, various configurations are available here. Also, we can configure settings for our brokers, which are listed here. As evident, there are numerous configurations, including types for users, clients, IPs, and more. I won’t go through the entire documentation but will demonstrate how to utilize it.

The basic command is:

kafka-configs --bootstrap-server localhost:9092 --entity-type topics --entity-name my-topic

And we have to choose between 2 options: describe to get dynamic configuration or alter for updating the dynamic configuration.

Like : — alter — add-config min.insync.replicas=2

kafka-configs --bootstrap-server localhost:9092 --entity-type topics --entity-name my-topic --alter --add-config min.insync.replicas=2

Minimum in-sync replicas have been updated to make sure we can do the same base command with the description.

kafka-configs --bootstrap-server localhost:9092 --entity-type topics --entity-name my-topic --describe

Don’t be confused kafka-configs … --desribe and kafka-topics --describe are the different commands!

Now delete our in-sync replicas configuration for it we have to add --alter -- delete-config <config-name>

kafka-configs --bootstrap-server localhost:9092 --entity-type topics --entity-name my-topic --alter --delete-config min.insync.replicas

Conclusion

• See all configs: kafka-topics ... --describe
• See only dynamic config: kafka-configs ... --describe
• Adding config: kafka-configs ... --alter --add-config <property>
• Delete config: kafka-configs ... --alter --delete-config <property>

Thank you for reading until the end. Before you go:

Apache Kafka At the Gates of Mastery: Kafka Core

• Please consider clapping and following the writer! 👏
• Follow us on Twitter(X), LinkedIn