Install Kafka on MacOS using brew

Photo by Lukas Blazek on Unsplash

Introduction

Kafka is an open-source, distributed, event-streaming platform, that helps applications process records of streams in a fast and durable way. It works as a broker for sending messages from one application to another. It helps in developing real-time event-driven applications.

And an Event can be considered as an activity or action which we do in an application, like placing an order, submitting a form etc. Each action has some data and time associated with it.

Kafka was developed by LinkedIn, later it became open-source. Confluent is a platform which uses Apache Kafka at its core.

Installation of Apache Kafka

For working with Kafka we need to install it in our system, so in this tutorial, I am going to explain how to install apache kafka in your system and then later will develop some apps using Kafka and spring boot.

Steps to install Kafka in macOS (Apple chip):

1. First, install Homebrew in your system if it is not in your system. homebrew is a package manager it installs required projects/libraries in your system easily.
   Go to: https://brew.sh/
2. After homebrew is ready now run the command from the terminal.
   brew install kafka
3. After this you will see this message in the console to restart Kafka after an upgrade:
   brew services restart kafka
4. Now for running Kafka first we need to start Zookeeper and then run Kafka. Command to start Zookeeper.
   zookeeper-server-start /opt/homebrew/etc/kafka/zookeeper.properties
5. After the zookeeper started then run the Kafka command
   kafka-server-start /opt/homebrew/etc/kafka/server.properties
6. If error-related to the connection on Kafka then open the server properties file:
   vim /opt/homebrew/etc/kafka/server.properties
   and replace:
   listeners=PLAINTEXT://:9092
   with
   listeners = listener_name://host_name:port
   and restart the server after that.
7. Now you can create a topic in Kafka and you can publish some messages on that topic using the commands.
   kafka-topics — create — bootstrap-server localhost:9092 — replication-factor 1 — partitions 1 — topic learn
8. The above command creates a topic named learn. now publish messages on learn topic, for that we need to initialize the producer console.
   kafka-console-producer — broker-list localhost:9092 — topic learn
9. Now to see these messages we need to initialise the consumer console.
   kafka-console-consumer — bootstrap-server localhost:9092 — topic learn — from-beginning
10. After running the above command in the new console you will able to see all the messages you have written in the producer console.

That’s all for this tutorial it will help you in setup Kafka in your local environment, after this you can learn all other topics related to Kafka.

Kafka is used by most companies so learning Kafka will help you in landing a good job as well, there are multiple use cases of Kafka that also you can learn.

Thanks for reading!!