Schemaregistry

When we start the schema registry, by default it provides us with the open REST APIs using which we can work on the data, schema definition, version manipulation operations etc. Below are few important and required schema registry manipulation curl commands, but not all of them are given in this…

To understand the basics of Schema registry, it’s behavior and the working you can refer here. …

Due the Apache Kafka’s publish subscribe architecture, scalable nature and fault tolerant ability, it has been a gaining popularity among the developers community. As we know, we have the producers and consumers in kafka which perform their jobs as their names suggests, Kafka generally sends data into byte format over…

Managing Kafka event schemas for 2000 microservices can be quite challenging and painful. Potential challenges include: How can you maintain safe schema evolution and avoid catastrophic poison pills that will bring down your Kafka consumers? How can you make schemas easily discoverable and avoid wasting precious dev time? How can…

Before deepening into the different supporting technologies, let’s create a baseline about schemas and message brokers or async server-server communication. Schema = Struct. The shape and format of a “message” are built and delivered between different applications/services/electronic entities. Schemas can be found in SQL & No SQL databases, in different…

My name is Nina and I work as a Golang Backend Software Engineer at a large retail company that has an online grocery delivery application. Recently, I was faced with a standard task of receiving a large amount of data asynchronously from another service. We chose to use Kafka broker…

I am continuing a series of articles about Kafka and Schema Registry. The first part can be read here: Part 1: Introduction and Go Producer for Kafka and Schema Registry Part 2: Go Consumer for Kafka and Schema Registry In the previous article, we used Schema Registry to produce a message to the Kafka broker. …

In this part, we will discuss how validation occurs during schema evolution. We will also consider what pitfalls you may encounter if you write a client for the producer yourself. First two parts could be found here: Part 1: Introduction and Go Producer for Kafka and Schema Registry and Part…

In today’s M2M (Machine to machine) communications landscape, there is a huge requirement for streaming the digital data from heterogeneous IoT devices to the various RDBMS for further analysis via the dashboard, triggering different events to perform numerous actions. To support the above scenarios Apache Kafka acts like a central…