- Start Zookeeper:
zookeeper-server-start.sh /path/to/zookeeper.properties- Start Kafka Broker:
kafka-server-start.sh /path/to/server.properties- Stop Zookeeper:
zookeeper-server-stop.sh- Stop Kafka Broker:
kafka-server-stop.shTopic Management
- Create a Topic:
kafka-topics.sh --create --topic my-topic --bootstrap-server localhost:9092 --partitions 3 --replication-factor 2- List All Topics:
kafka-topics.sh --list --bootstrap-server localhost:9092- Describe a Topic:
kafka-topics.sh --describe --topic my-topic --bootstrap-server localhost:9092- Delete a Topic:
kafka-topics.sh --delete --topic my-topic --bootstrap-server localhost:9092- Alter a Topic (e.g., Increase Partitions):
kafka-topics.sh --alter --topic my-topic --partitions 5 --bootstrap-server localhost:9092Producer Commands
- Start a Console Producer:
kafka-console-producer.sh --broker-list localhost:9092 --topic my-topic- Start a Console Producer with Keyed Messages:
kafka-console-producer.sh --broker-list localhost:9092 --topic my-topic --property "parse.key=true" --property "key.separator=:"- Send a Message: After running the console producer command, you can type:
message_value- Send a Keyed Message: After running the keyed console producer command, type:
key1:message_value1Consumer Commands
- Start a Console Consumer:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my-topic --from-beginning- Start a Console Consumer with Group ID:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my-topic --group my-group- Consume Only Keyed Messages:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my-topic --property print.key=true --property key.separator=":"- Consume Messages from Specific Offset:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my-topic --offset 10 --partition 0Consumer Group Management
- List All Consumer Groups:
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list- Describe a Consumer Group:
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group- Reset Consumer Group Offset to the Earliest:
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-group --reset-offsets --to-earliest --topic my-topic --execute- Reset Consumer Group Offset to the Latest:
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-group --reset-offsets --to-latest --topic my-topic --execute- Delete a Consumer Group:
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --delete --group my-groupMessage Offsets
- Get Earliest Offset for a Partition:
kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic my-topic --time -2 --offsets 1- Get Latest Offset for a Partition:
kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic my-topic --time -1 --offsets 1Cluster Management
- Check Cluster Health:
kafka-run-class.sh kafka.admin.BrokerApiVersionsCommand --bootstrap-server localhost:9092- List All Brokers:
zookeeper-shell.sh localhost:2181 <<< "ls /brokers/ids"- Describe Broker Configurations:
kafka-configs.sh --describe --entity-type brokers --entity-name broker-id --bootstrap-server localhost:9092Partitions and Replication
- Reassign Partitions:
kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --reassignment-json-file reassignment.json --execute- List Under-Replicated Partitions:
kafka-topics.sh --describe --under-replicated-partitions --bootstrap-server localhost:9092Kafka ACL (Access Control Lists)
- List ACLs:
kafka-acls.sh --list --bootstrap-server localhost:9092- Add ACL for a User to Read from a Topic:
kafka-acls.sh --add --allow-principal User:user --operation Read --topic my-topic --group my-group --bootstrap-server localhost:9092- Remove ACL for a User:
kafka-acls.sh --remove --allow-principal User:user --operation Read --topic my-topic --bootstrap-server localhost:9092Log Management
- Dump Log Segments:
kafka-run-class.sh kafka.tools.DumpLogSegments --files /path/to/logfile --print-data-logMonitoring and Metrics
- Get Broker Metrics:
kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi- Enable JMX Monitoring (Start Kafka Broker with JMX enabled):
JMX_PORT=9999 kafka-server-start.sh /path/to/server.propertiesReplication
- Describe Replicas:
kafka-topics.sh --describe --topic my-topic --bootstrap-server localhost:9092- List Offline Partitions:
kafka-topics.sh --describe --bootstrap-server localhost:9092 --under-replicated-partitionsAdvanced Tools
- Mirror Maker (Replicate data between Kafka clusters):
kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config consumer.properties --producer.config producer.properties --whitelist 'my-topic'- Run Kafka in Single-Node Mode:
kafka-server-start.sh /path/to/config/server.properties --override listeners=PLAINTEXT://localhost:9092Kafka Connect
- Start Kafka Connect:
connect-distributed.sh /path/to/connect-distributed.properties- List Kafka Connect Connectors:
curl -X GET http://localhost:8083/connectors- Deploy a New Connector:
curl -X POST -H "Content-Type: application/json" -d @connector-config.json http://localhost:8083/connectors- Delete a Connector:
curl -X DELETE http://localhost:8083/connectors/my-connectorKafka Streams
- Run Kafka Streams Application:
kafka-run-class.sh com.example.MyKafkaStreamsApp- Inspect Kafka Streams Application State:
kafka-streams-application-reset.sh --application-id my-streams-app --bootstrap-servers localhost:9092This cheat sheet covers the most common and advanced Kafka commands, from setting up and managing topics, producers, and consumers to managing ACLs, replication, and monitoring Kafka clusters. It’s a handy reference for Kafka users of all levels.
