Adopting a Consumer-First Mindset When Building your Data Mesh
One expert shares how focusing on the human element of data mesh can help teams avoid some common mistakes
Over the past couple of years, data mesh has become an increasingly important topic for executive decision makers. My colleague Miranda LeQuire has written an excellent piece summarizing why data mesh is important for organizations as we deal with uncertain economic conditions in a post-pandemic world.
In this post we are going to focus on some practical considerations and lessons from mesh implementations. As you go through your mesh journey, it’s important to remember the actual business value of the data mesh lies in the ability of people to consume data products and use them to drive decisions.
A great way to organize the data mesh journey is to structure it into four phases: Align, Build, Govern, and Adopt. Below we talk about some of the key things to watch out for in each phase to ensure that data consumption is a focus throughout.
Align
This is the stage where you decide whether data mesh makes sense for your organization. The most important thing to remember here is that moving to a data mesh is a technology AND a business change. While the technology part is usually clear to everyone, the business engagement aspect is often underestimated or sometimes completely overlooked. This usually stems from a history of technology teams owning the end-to-end journey of data products.
When we work with our clients, we start by getting support from company executives and defining the vision and goals. Getting deeper and defining metrics to measure success toward the established goals ensures we drive the right behaviors to facilitate a consumer-first mindset (e.g., time to access data, data product ratings).
With senior leadership on board, it’s easier to start working on the next level: identifying different business domains and domain owners within the organization (tip: involve your architecture/backend services team and see if they have already defined bounded contexts or domains from a services perspective that would work for your mesh). With the support of senior leadership, initiate conversations with domain owners and evangelize the benefits of a data mesh. At the same time, use these conversations to get a better understanding of the capabilities within a given business domain. Understanding where there might be gaps (in skills, product mindset, etc.) will come in handy later as you try to encourage adoption within the organization.
Build
Teams creating data products need underlying infrastructure to work with that data, all the way from ingesting data to processing and sharing it. While data mesh doesn’t enforce any specific technology, for different data teams to be nimble and rapid in their execution, organizations need to build a self-service data platform/infrastructure, which allows different domains to rapidly build out their data products.
Simple as it sounds, a user journey–mapping exercise to explore how different personas work within the current state provides you with a clear set of capabilities that your new platform needs to provide for your consumers. As an example, in the diagram below, to enable a data scientist who is looking to deploy a machine learning model, the platform team could help build blueprints that start with provisioning a notebook and end at having the model deployed in production. These blueprints not only accelerate development of data products but also provide a layer of governance with teams using secure and vetted patterns for their development.
As important as these blueprints are, don’t fall into the classic trap of letting your central team become a bottleneck. For example, if your risk team wants to deploy a fraud detection model on AWS SageMaker and your platform doesn’t (yet) have a blueprint, don’t let that be a holdup. In fact, allow your risk team to build a reusable blueprint and contribute it to the platform team’s resources, which others can use.
Another common mistake during the Build stage is to focus too much on enabling data producers and not paying enough attention to data consumption. Too many times, very little up-front thought is put into how people or teams can discover and get access to data. This usually results in painful rework later, plus lower and/or delayed ROI. Defining the access models and specifically deciding how granular you want to be in providing data access (column, table, domain access, etc.) and on the method of providing data (APIs, SQL, etc.) in advance is essential!
Govern
One of the big challenges when moving from a centralized structure to a data mesh can be a vacuum in governance. Each team is building things the way they want, with an architecture they think best, and sharing data products in a manner they see fit. How then do you ensure that data being produced by each team meets quality criteria, adheres to data protection practices, and is advertised and shared in a manner that is consistent with other teams within the organization?
While you want teams to maintain autonomy over certain decisions (e.g., how to model the data, choice of tools, etc.), you need to ensure adherence to global standards (e.g., data products need to be consumable in an agreed format, data products must be published to a catalog, etc.). Having a central governance council can help ensure consistency without becoming a blocker to progress. Typically, this council would be run by a central governance team, with members from each domain providing input AND being accountable for their domain’s compliance.
In keeping with a consumer-first mindset, the central governance team also needs to provide shared tooling (e.g., enterprise data catalog) and templates to prevent siloed implementation of standards and to ensure standardization across the organization, resulting in discoverable and quality data for consumers.
Adopt
Last and arguably most important, adoption. A successful data mesh requires fundamental changes to culture, ways of working, and even organizational structure. Many organizations underestimate the importance of managing this change. Invariably, this leads to a lower ROI than anticipated at the start of this journey.
The behaviors of stakeholders (executives, product owners, business analysts, developers, etc.) within the organization are different by role. Understanding how people work today allows you to understand what will be needed to move them to a new way of working and ensure a sustained change. This goes back to the journey mapping we discussed earlier — knowing how internal and external consumers interact with your data mesh, and how that differs from what they do today.
Remember, people within your organization will have very human reactions to the changes a data mesh brings. While some will be enthusiastic supporters of the change, a majority will likely be passive OR detractors. Teams and individuals need to be incentivized to promote and accelerate adoption. Bringing them along for the journey, from the very beginning, and developing quick wins in the form of use cases where a data product can be utilized is great to build momentum and ignites a desire in people to replicate that success.
In addition, the talent within your organization will need to evolve and grow to support the future state of data mesh. Utilizing a coaching approach to help your people understand the new capabilities and growth opportunities will gain excitement around change. A model that works well is to temporarily embed members of the central team into the different domain teams. Aside from helping the data product team, it helps the central team get a better understanding of domain needs too.
If you can spare 30 minutes, check out this talk from a recent Datanova event held by Starburst, which digs deeper into managing change when setting up your data mesh.
Conclusion
Looking back at the diagram depicting the four stages of a data mesh implementation, note that your journey hasn’t ended with the “Adopt” stage. As long as you are building new data products and new capabilities, that wheel is going to keep on spinning!
In the end, remember the vision and goals you set at the start of this journey. The data mesh is just a means to that end, not the end itself. If you can remember that throughout the journey, a big part of the job is already done!
Slalom is a global consulting firm that helps people and organizations dream bigger, move faster, and build better tomorrows for all. Learn more and reach out today.
