The State of Data Infrastructure Landscape in 2022 and Beyond

Key trends to expect in the data infrastructure domain in 2022 and beyond.

Photo by Cayetano Gil on Unsplash

The decade from 2010 to 2020

Source

The decade from 2020 to 2030

Credits — Prukalpa Sankar

The Modern Data Stack

Modern Data Stack is a radically new approach to data integration that saves engineering time, allowing engineers and analysts to pursue higher-value activities.

Charles Wang, Fivetran

MDS components at a minimum

Analytics engineering and dbt

Analytics engineers provide clean data sets to end users, modeling data in a way that empowers end users to answer their questions. While data analysts spend their time analyzing data, analytics engineers spend their time transforming, testing, deploying, and documenting data. Analytics engineers apply software engineering best practices like version control and continuous integration to the analytics code base.

Claire Carrol, getdbt.com

Analytics engineer sits between data engineer and data analyst.

Streaming databases/low-latency OLAP databases

Incremental updated materialized view engines

Real-time OLAP databases with the scatter-gather query execution

Classification of streaming databases

Metadata management and data catalogs

A data catalog creates and maintains an inventory of data assets through the discovery, description, and organization of distributed datasets. The data catalog provides context to enable data stewards, data/business analysts, data engineers, data scientists, and other line of business (LOB) data consumers to find and understand relevant datasets to extract business value.

— Gartner, Augmented Data Catalogs 2019

Data lakehouses and open data architecture

Source

Data platform as a service (dPaaS)

Conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dunith Dhanushka

Editor of Event-driven Utopia(eventdrivenutopia.com). Technologist, Writer, Senior Developer Advocate at Redpanda. Event-driven Architecture, DataInMotion