Apache Kafka Guide #41 Topics Naming Convention

Apache Kafka Guide #40 Topics Naming Convention

Hi, this is Paul, and welcome to the #41 part of my Apache Kafka guide. Today we will discuss the Topic naming convention.

Apache Kafka At the Gates of Mastery: Kafka Core

Naming Conventions Guideline for Topics

<message type>.<dataset name>.<data name>.<data format> - with snake_case

Message Type (<message type>):

• user: For user-specific data such as scratch and test topics.
• queuing: For classical queuing use cases.
• logging: For logging data (slf4j, log4j, etc)
• etl/db: For CDC and ETL-like database feeds.
• streaming: For intermediate topics created by stream processing pipelines.
• push: For transferring data from offline (batch processing) environments to online systems.
• tracking: For tracking events such as page views, user clicks, etc.

Dataset Name (<dataset name>)

• The dataset name functions similarly to a database name in conventional RDBMS systems, serving as a category to organize topics under one umbrella.

Data name (<data name>)

• The data name field corresponds to a table name in conventional RDBMS systems. However, developers are allowed to use additional dotted notation to create a hierarchy within the dataset namespace if desired.

Data format (<data format>)

• The data format for example .json, .text, .avro, .protobuf, .csv, .log

Thank you for reading until the end. Before you go:

Apache Kafka At the Gates of Mastery: Kafka Core

• Please consider clapping and following the writer! 👏
• Follow us on Twitter(X), LinkedIn

Paul Ravvich