Data Mesh: Reconfigure your enterprise data analytics team to become value multipliers.
Gone are the days where data engineering and data analytics can only be done by the few in the centralized IT-data team in the company. The business data users have been upskilling — BI, Python, Statistics and more. Data analytics buzz have required folks in other IT departments to dabble deeper into Data/ML/AI requirements too. Business software are also incorporating data analytics features as part of their full suite offering.
By now, there are probably 2 kinds of operating models in the enterprise for data analytics: (1) Factory Line, (2) Self-Serve.
Operating Model #1. One-way factory line from data provider to data analytics team to data user
In this operating model, the data from data providers are aggregated and analyzed by a centralized data analytics team to produce ‘Data Products’ that are pushed to data users according to their ask. Over time, the data analytics team could get really good at producing consistent data analytics value with standard operating procedures, quality framework, tools and team members becoming experienced. On the other hand, a steady operating factory line also means it is harder to configure change — everyone is afraid of factory line breakdown and unable to fulfil service level agreement with business data users.
Operating Model #2. One-way self-produced and self-consumed by any data users
This operating model is sometimes known as self-service analytics and is grown out of frustration of depending on the centralized data analytics team, or the belief that the business domain owner can do better analytics. The existent of this operating model also represents a growing data analytics maturity in the company where data analytics value is self-produced and self-consumed by any one in the company. The centralized data analytics team would then need to play as a support or consultative or trainer role on top of the work in operating model pattern #1 — e.g. how-to do a certain task using BI tool X, how to connect to data mart Y, etc.
With more users across the company becoming more proficient in data analytics, the demand for quicker turnaround, better self-service tools, deeper analytical expertise in business and technical domain will definitely place a higher stress on the centralized data analytics team in the company. On the other hand, the centralized data analytics team responded by focusing on getting bigger, using better ETL/ELT tools, database systems, algorithms.. for themselves to become better, faster at doing what they are doing now but never fast enough to meet demand.
This is actually the onset of centralized data analytics function — the team and platform becoming monolithic and the factory line becomes sluggish in producing quality value.
Meet the new centralized data analytics team that doesn’t analyze business data.. sort of.
The renewed role of this team is to act as Facilitators, Coaches, Guides, Encouragers, Platform Business Owner with a sole mission to multiply data analytics value across the company. Here is 1 proposal to illustrate how this could be implemented in the company:
Mobile Experts: Mobilize the best data engineers and data scientists from the centralized data analytics team to transfer into the 2 to 3 business function who are the key data providers, to jumpstart. The business functions are already domain experts. Equipped with data analytics capability, the entire function would become more critical of their own data and data quality. The refreshed business function could look forward to quicker time to domain insights, and having dedicated data specialists to liaise all data-related queries within and outside of the business function, or even external parties.
Platform Growth Team: The old way of sending requirements to the monolithic data analytics team needs to be slowly shed off. Pick the best business analysts and data analysts from the original centralized data analytics team to help preach the new ways of working, helping individual business functions to be aware of data available in other functions, prepare demos and help guide the growth of data analytics value across the company. This team is a core engine to seek new partnerships internal and external, generate demand, sign-ups, conversion and retention of internal and even external users by curating data and analytics products and providing consulting and advisory, events and trainings.
Platform Products Team: Instead of leading ‘Data Products’ for the business function, this team looks at platform-level products. For example, helping data producers to hire data experts, data app stores and catalog, designing reusable data architectural patterns for the refreshed business teams to build faster, standardized data and analytics service APIs for external developers, UX, monetization and more. The platform product team could be made up of previous data leaders, product managers, and data analytics architects.
Platform Engineering Team: Implements the platform products and provides operation support for the platform products (not the data products produced by business functions)
Platform Governance Team: Design metrics and conduct measurement of platform participants, platform products and interaction. This team work with the other platform teams to continuously experiment the ways to ensure platform usefulness and longevity in the enterprise, for example:
1) Pull, match and facilitate interactions amongst producers and users;
2) Source or create required funding and cashflow;
3) Drive innovation, prioritize new functionalities and setting necessary guardrails to reduce wastages.
With this reconfiguration, individual business and IT teams become data analytics experts of their domain area. The new platform teams (Governance, Engineering, Products, Growth) further empower them to extend their data analytics knowledge and operations by continuously curating, quality control, pull, match and facilitate data analytics products amongst internal and external teams, multiplying data analytics across the enterprise and facilitating new social interaction activities.
This gives rise to operating model #3 of data and analytics in the enterprise — Distributed, multi-ways production and consumption of data and analytics products and value.
To be updated..
But, what do you think?
