A Gentle Introduction to Event-driven Change Data Capture

How to detect, capture, and propagate changes in source databases to target systems in real-time, event-driven manner

A Rube Goldberg Machine — Credits

The motivation behind CDC

Application maintains its data in multiple places

Systems of Records and derived data

Source data and derived data need to be synchronised

Change Data Capture (CDC)

Change detection methods

How does a log-based CDC system work?

Theoretical view of a CDC system

Requirements for a production-grade CDC system

Making it event-driven

A real-time, event-driven CDC system architecture

1. Change event generation

Maxwell’s event format — Source

2. Change event ingestion

3. Change event propagation

Advantages

Use cases

Tools in the market

Debezium

Debezium Architecture — Source

Maxwell

Conclusion

References

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dunith Dhanushka

Editor of Event-driven Utopia(eventdrivenutopia.com). Technologist, Writer, Senior Developer Advocate at Redpanda. Event-driven Architecture, DataInMotion