Understanding Kafka Topic Partitions

Everything in Kafka is modeled around partitions. They rule Kafka’s storage, scalability, replication, and message movement.

Dunith Danushka
Tributary Data

--

Photo by Meggyn Pomerleau on Unsplash

P.S I did some edits to reflect the feedback I received from the audience. Thanks for your valuable contribution. I expect more! :)

Everything in Kafka revolves around partitions. They play a crucial role in structuring Kafka’s storage and the production and consumption of messages.

Understanding partitions helps you learn Kafka faster. This article walks through the concepts, structure, and behavior of Kafka’s partitions.

Events, Streams, and Kafka Topics

Before diving into partitions, we need to set the stage here. So let’s look at some high-level concepts and how they relate to partitions.

Events

An event represents a fact that happened in the past. Events are immutable and never stay in one place. They always travel from one system to another system, carrying the state changes that happened.

Streams

An event stream represents related events in motion.

Topics

--

--

Dunith Danushka
Tributary Data

Editor of Tributary Data. Technologist, Writer, Senior Developer Advocate at Redpanda. Opinions are my own.