Kafka Schema Registry & Avro: Introduction

This article is the first in a series of articles on the schema registry and Avro serialization. It provides an overview of Avro and the schema registry, and examines the Avro serialization and deserialization flows in the Kafka consumer and producer.

Messages being written to and consumed from a topic can contain any data as they are byte arrays. By applying a schema it ensures that the message written to a topic by a producer can be read and understood by a consumer of the topic. In order to manage and make such a schema available, a schema registry is used that both the producer and the consumer talk utilise. One such message schema is provided Apache Avro, a serialisation framework which provides type handling and compatibility between versions.

The source code for the accompanying Spring Boot demo application is available here.

See the ‘More On Kafka Schema Registry & Avro’ below for more articles in this series, including covering the accompanying Spring Boot demo.

The Importance of a Schema

Schemas for Kafka messages are optional, but there are strong benefits for choosing to use them. A schema defines the fields and types of the message ensuring that only those messages that meet this definition will be understandable by the serializer or…