Unbundling the Modern Streaming Stack

Why modern streaming stack is replacing the classic streaming architecture? What’s the composition and what values it brings?

Photo by Phil Goodwin on Unsplash

TL;DR

What is a streaming stack?

Streaming data: events and event streams

An event carries state changes from event producers to consumers
A stream of events

Streaming analytics

What is the “classic” streaming stack?

Lambda architecture

Why classic streaming stack didn’t pickup?

What is the modern streaming stack?

MSS is influenced by Kappa architecture

Kappa architecture — Source

MSS is built with modern, cloud-native technologies

MSS goes beyond the JVM and addresses other languages

Developer friendliness and rich tooling

Unbundling the modern streaming stack

The modern streaming stack

Event streaming platform

Real-time data ingestion layer

Stream processing layer

Serving layer

Data products and APIs

Tiered storage

Metadata management, schema evolution, and data governance

What’s coming next

References

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dunith Dhanushka

Editor of Event-driven Utopia(eventdrivenutopia.com). Technologist, Writer, Senior Developer Advocate at Redpanda. Event-driven Architecture, DataInMotion