Geek Culture
Published in

Geek Culture

The 2022 Data World in Three Words: “I Was Wrong” — 6 Truths I didn’t understand 365 days ago.

Photo by Michal Matlon on Unsplash, modified by the author.

(1) The Data Mesh is there to stay, but it’s not coming as fast as expected.

(2) “Just do what the software engineers do” isn’t going to cut it in the data space.

  1. Data people still have largely a different background, culture & practices than software engineers
  2. The data value creation process looks different than the software one
  1. The data mesh as “just another decentralization movement” makes sense only, if the company culture supports it.
  2. Machine learning integrated into products only makes sense, if the company has product management that openly embraces this.
  3. If your company considers data to be a sidekick, then no big investment in best practices makes sense.
  4. Pushing data into your CI system makes sense only as long as it stays small. Yes, it’s called “dbt — data build tool” — but it is not the same as a software build process, not for the vast majority of cases.

(3) 99,9% of companies are not aware of what data is important and what to do with it.

  1. I realized more kinds of data are important than I believed to be before (see below)
  2. A few major trends actually moved the world in the opposite direction!
  1. Real-time data use (not to be confused with event-driven architectures)
  2. Unstructured data (and everything that builds on top of a use case)
  3. Company external data (and everything that builds on top of as a use case)
  4. Turning data into actions

(4) Even a collapsed crypto world has so much to teach the data world.

Photo by Kanchanara on Unsplash.
  1. Systems thinking tops product thinking.
  2. Community-led innovation tops company-internal innovation in terms of breadth.
  3. Open tops closed.

(5) OS is hard, like really hard to pull off. And nobody talks about it.

(6) Open Source is not the only option for data companies, but key challenges need to be solved.

  1. some kind of standard/ protocol,
  2. on a large degree of openness,
  3. and need to make network effects key to your company strategy.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sven Balnojan

DataOps @ Meltano | Data PM | co-author of “Data Mesh in Action” | Join my free data newsletter “Three Data Point Thursday” at http://thdpth.com/