A Gentle Introduction to Event-driven Change Data Capture
How to detect, capture, and propagate changes in source databases to target systems in real-time, event-driven manner
This post serves as an introduction to the Change Data Capture (CDC) practice, rather than a deep-dive on a particular tool. First, I will explore the motivation behind CDC and illustrate the components of a real-time event-driven CDC system. The latter parts discuss some potential use cases where CDC is applicable and conclude with some open-source tools available in the market.
P.S I made a YouTube videos for this post to explain this visually as well. If you are interested, you can check it below.
The motivation behind CDC
Applications start with a small data footprint. Initially, a single database fulfils every data need of the application.
When applications evolve, they need to support different data models and data access patterns. For example, they might need a search index to perform…
