Data Mesh — Benefits
Technology Perspective
This article analyzes the potential lying in Data Mesh implementation as seen through the eyes of the technologist. It is an extract from “Data Mesh in Action” by Manning Publications, the first book on the implementation of the Data Mesh paradigm, which I co-author together with Jacek Majchrzak, Sven Balnojan, Mariusz Sieraczkiewicz.
Data Mesh definition and need
The Data Mesh is a decentralization paradigm. It decentralizes the ownership of data, the transformation of data into information, and data serving. A more thorough definition of Data Mesh can be found in this article.
We believe that the data world is in need of decentralization of data in the form of the Data Mesh. Some of the reasons were mentioned previously. However, Data Mesh is not always the best solution. In many cases, centralization may be the best option, as exemplified here.
Data Mesh speeds up development
The main benefit of Data Mesh from the technological perspective is keeping the speed of development with the organization’s growth. Data Mesh is meant to address the shortcomings of other data architectures, like Data Warehouses or Data Lakes, by decentralizing data production and governance.
Those architectures introduce a bottleneck — a central team responsible for harmonizing all the data for the whole of your company and making it ready for consumption. A single team cannot scale to accommodate the varied data needs of a growing organization. Both the technology as well as the team knowledge quickly becomes a scale problem. Eventually, more time is spent on maintenance, and new projects become more and more delayed.
Data Mesh clarifies data ownership
The other benefit of Data Mesh is the clarity of data ownership right from the point of its creation. It flattens the data management structure leaving just a thin layer of a Federated Governance Team. And even that team’s activities are limited to agreeing on standards within autonomous domains.
Data Mesh empowers teams
The increase in the speed of development also comes from empowering the implementation teams. Since producing and maintaining the Data Products lies on their shoulders, the speed of change is not limited by a single central integrations team’s backlog of tasks. It means that both the evolution of and fixes to the Data Product happen quicker. This is especially prominent in case of any bug fixing and downtimes.
Furthermore, the team that owns a given Data Product is better equipped to react faster because there is no context switching, as is the case with a single central data team.
Data Mesh creates robustness
The other factor worth mentioning is data environment stability. With Data Products offering access to contracted versions of its datasets, pipelines built on them are much more robust and require much less maintenance.
To read about Data Mesh benefits as seen from the business perspective, check out this article.
This was an extract from “Data Mesh in Action” by Manning Publications.
To learn more about Data Mesh and the book check this episode of the “Between Data & Risk” podcast, which I host together with Artur Guja.
