Event Driven Architecture For Business

Credit Unsplash

Looking at modern data-engineering works sometimes feels boring. All live-in files, code, and version control system.

There are no fancy user interfaces, complex charts, Excel manipulations, or oral decisions.

Everything is traced.

Every data computation. Every step. Every decision.

As data scientists, engineers or analysts, we often wonder how our work is used to drive decisions.

The end of the data pipeline often lives in slide decks or dashboards, but decisions beyond usually have no shapes.

Problems With Dashboard Driven Decisions

Dashboards are probably good. Not that much for business.

We are building an infinite number of dashboards like it’s the end product of any business questions. It’s not.

Dashboards grow the set of questions. They allow exploration and quick insights discovery.

But there are no traces of how people took decisions after looking at dashboards.

Furthermore, the lifecycle of dashboards is often short. Among data teams, we often hear “it will never be used at the end”. There is always back and forth between slides producers and users. It adds friction and time costs.

Self-serve analytics tools such as dashboards make it possible for anyone to extract and visualize data. Anyway, they don’t imbue their users with the skills required to make sense of data and all its complications. Serving insights as a service would be complete if we also teach business people about confounding, bias, and outliers.

Without those skills, data’s not that useful.

This is why reports or PowerPoint slides are sometimes more useful to answer business questions.

First, it’s often produced by a data analyst. It comes with more control, more data literacy and a better scope of what’s is possible or not.

Data analysts also have domain knowledge (and sometimes data analysts don’t work directly within business units…).

When released, the report can be showed to stakeholders during meetings. We can trace document updates and decisions…