Getting Started with Kafka

Kafka with Docker on your local machine

I’ve set up Kafka on Docker and Docker-Compose for myself and other folks to use on their local laptops.

Installing Docker

To get started, you’ll first need to download and install Docker.

• Create an account at: https://hub.docker.com/editions/community/docker-ce-desktop-mac
• Download and install Docker Community Edition

Now that you’re signed in, head to your terminal and check that the CLI is installed with:

docker version

Running Kafka in Docker

Next, git clone my kafka-standalone docker setup.

git clone https://github.com/Kotlin-Thursdays/kafka-standalone.git
cd kafka-standalone
docker-compose up -d

This will download and spin up the necessary docker containers to run kafka on your local laptop. This may take a few minutes, depending on your internet connection and laptop speed. When it’s finished, you can check that it’s running correctly:

docker-compose ps

Creating A Topic

You can create a new topic in Kafka to send and receive messages on:

docker-compose run --rm kafka-client /kafka/bin/kafka-topics.sh --zookeeper zookeeper:2181 --create --topic messages --partitions 1 --replication-factor 1

Sending Messages on Kafka

You can spin up a producer to send messages with:

docker-compose run --rm kafka-client /kafka/bin/kafka-console-producer.sh --broker-list broker:29092 --topic messages

Once that spins up, you should be able to type messages and send them by hitting the enter key:

> This is a test message

Receiving Messages on Kafka

You can spin up a consumer to receive messages:

docker-compose run --rm kafka-client /kafka/bin/kafka-console-consumer.sh --bootstrap-server broker:29092 --topic messages --from-beginning

You should see your message show up:

This is a test message

That’s about it! You’re ready to start building applications with Kafka!

Building an Order System with Kotlin, Kafka, and Domain-Driven Design

Kafka is a scalable event-driven message system able to communicate in intense and complex systems. In the next post, we’ll be demonstrating the power of Kafka by building an online-ordering system!