Data Mesh — Benefits
Business Perspective
This article analyzes the potential lying in Data Mesh implementation as seen through the eyes of the business decision-makers. It is an extract from “Data Mesh in Action” by Manning Publications, the first book on the implementation of the Data Mesh paradigm, which I co-author together with Jacek Majchrzak, Sven Balnojan, Mariusz Sieraczkiewicz.
Data Mesh definition and need
The Data Mesh is a decentralization paradigm. It decentralizes the ownership of data, the transformation of data into information, and data serving. A more thorough definition of Data Mesh can be found in this article.
We believe that the data world is in need of decentralization of data in the form of the Data Mesh. Some of the reasons were mentioned previously. However, Data Mesh is not always the best solution. In many cases, centralization may be the best option, as exemplified here.
The business perspective on data
From the business perspective, data itself is of little value. Worse, it means incurred costs! Sounds like heresy? To understand that statement, and if needed, convey it to your business partners, you need to understand the different levels in how people perceive reality.
A good approximation of this phenomenon is the so-called DIKW pyramid, derived from the 1934 play “The Rock” by T.S. Eliot. It represents data, information, knowledge, and wisdom as a hierarchical structure, where each next element can be derived from the former.
The data in this context is just a set of values (which costs money to store). To derive value from it, one needs to build up the context allowing for informed decision-making. The Data Mesh improves the robustness of the whole pyramid.
Data accessibility and completeness
As we mentioned, having the raw data is of no use to decision-makers. One can argue that they can download it to their laptops and analyze it themselves. It is true! It has, however, two underlying assumptions:
- To download the data, it needs to be accessible.
- To ensure the value of any performed analysis, data needs to be as complete as possible.
Let us address the first assumption. We mentioned already and will say repeatedly, Data Mesh is very much focused on making data accessible. Not only accessible but findable, interoperable, and reusable as well! This is embedded in one of the four Data Mesh principles — Data as a Product — which is all about making sure data is there for the taking.
Completeness of the data is another issue where Data Mesh shines. Unlike most Data Warehouse or Data Lake architectures, Data Products and their data models are not developed by IT specialists in isolation from business concerns. Instead, it is a joint effort, ensuring the data presented outside the domain is sufficient to derive meaningful conclusions.
Data Mesh adds value
Data Mesh also helps add value to elements higher in the hierarchy. The teams transforming data into information, knowledge, and wisdom, which the business environment likes to call “insight,” gain instant access to multiple interoperable data sources.
Of course, in theory, it is possible to make it happen in a Data Lake as well. However, the reality of a single team managing technical aspects of the environment, as well as data access and transfer rights, is not feasible in our experience. And if required bits of data are stored in two different Data Lakes (or four, which is not that unusual), getting them all to work together is next to impossible.
In short, having access to read-optimized Data Products enables quick prototyping of new analytical methods and opens a path for the rapid development of new business capabilities.
To read about Data Mesh benefits as seen from the perspective of the technologist check out this article.
This was an extract from “Data Mesh in Action” by Manning Publications.
To learn more about Data Mesh and the book check this episode of the “Between Data & Risk” podcast, which I host together with Artur Guja.
