Add Schema Registry to Kafka in Your Local Docker Environment

Schema Registry helps maintain the contract between producer and consumer message data structures in Kafka topics

billydharmawan
Feb 18 · 6 min read
Photo by Dmitry Ratushny on Unsplash

Introduction

In the world of event-driven applications, one key thing is to make sure that the messages that get published to a topic or a queue can be understood by all the players, i.e. the producers and consumers.


Why Schema Registry?

We all understand that our Kafka producers publish messages to Kafka topics and our Kafka consumers read the messages from those topics. Schema Registry acts as a third person that ensures the messages published by the producers can be read by the consumers.


Add Schema Registry and Control Center Images to docker-compose File

We are going to add the Docker images for Schema Registry and Control Center to our existing docker-compose.yml file.

  • confluentinc/cp-enterprise-control-center:5.4.0
docker-compose.yml

Spin Up Docker Containers

Go to your terminal and go to the directory that contains the docker-compose.yml file. Let’s start up all the containers so we can play around with them.


Create a Topic

In the previous tutorial, mentioned in the introduction, we created the topic via the Kafka CLI tool. But now, we are going to use the Confluent Control Center to create the topic as it is much easier and quicker.


Define the Topic Schema

Key schema

Let’s start by defining our message’s key schema. Generally, we want our key to be unique to benefit the most from Kafka’s topic partition. Remember that the Leader will place the message in a partition based on the message’s key.

Value schema

Now, we will define the schema for the message’s value. This one is more interesting because we will see the beauty of the Avro schema.

  • The name of the assignee or who is responsible for getting it done.
  • The estimated duration to complete the task.
  • The timestamp at which the task was added to the to-do list.

Wrap Up

By the time we reach this section, we will have gained a basic understanding of how Kafka and Schema Registry work hand-in-hand. We also have seen how convenient it is to use the Confluent Control Center to manage our Kafka and Schema Registry.


Better Programming

Advice for programmers.

billydharmawan

Written by

A passionate Software Engineer trying to leave a good legacy on earth

Better Programming

Advice for programmers.

More From Medium

More from Better Programming

More from Better Programming

More from Better Programming

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade