Why and How to Run Data Teams as Product Teams?
Data teams often work as support functions in organizations. Their existence is misunderstood and misinterpreted that causing all the stakeholders only interact with them when they have problems, like IT teams; e.g. central KPI dashboards not updated in the last two days, sales numbers not matching in two reports, exporting customer account deletion requests in CSV asap, etc.
This process increases the knowledge, collaboration, and alignment gap between business and data teams every single day. When business units can’t predict that most of the problems can be solved by data products, big opportunities are missed. As a consequence data teams miss to achieve their potential.
In this article, I will illustrate why and how to run data teams as product teams. But first, we need to learn some basics about product management terminologies!
Product 101
What is product-led thinking?
Product-led thinking is a product development approach that prioritizes the user experience and the value that the product delivers to the customer.
With product-led thinking, the focus is on creating a product that is so compelling and easy to use that it naturally drives adoption and growth. This can involve designing a product with a self-serve onboarding process, providing education and support, and continuously improving the product based on customer feedback.
What is a product team?
A product team is a cross-functional group of individuals responsible for developing and launching a product. According to Marty Cagan at Empowered, the product team includes members from software engineering, product design, and product management.
Depending on the scope of the team, user experience (UX), product marketing, content localization, etc team members can be added to the product teams.
What is the purpose of the product team?
The purpose of a product team is to work together to understand customer needs, discover initiatives to address these needs, prioritize product initiatives to identify the most impactful ones, validate the product initiatives, create a product roadmap to develop the initiatives, design the user experience, develop the product, bring it to market, evaluate the results, provide maintenance, improvement, and evolution of the product in a non-ending loop.
Why should the data teams be managed as Product Teams?
Below I share the common problems of the data teams with examples of how they are treated in product teams.
The stakeholders are in the driver’s seat
Data teams
Have you ever been on a road trip as a passenger where the car is driven by someone else and you have no idea about the trip at all; where you are heading, how long it will take, how it will end, who the driver is, why did they take this route, what they want to achieve, etc. You have millions of questions but the driver doesn’t have a clear answer. Good luck and enjoy your trip!
Being in a service-oriented or support-oriented data team is precisely the same feeling. Your stakeholders pull you into meetings with hundreds of questions and tasks which all are important and urgent for the business, from their perspective. After those meetings, you get extra requests through Slack, email, or Jira tickets. Of course, they are also urgent and should be covered asap. You can’t understand the root cause of these requests or how they prioritize initiative X over Y.
Product teams
Successful product teams run continuous discovery sessions with their stakeholders and users to understand the problems, and opportunities to be foundations of their roadmaps. They never work as feature factories led by stakeholders.
Technical debts are not just for software engineers
Data teams
Technical debt describes what results when development teams take actions to expedite the delivery of a piece of functionality or a project which later needs to be refactored.
In software or data product development, leaders make decisions at speed over quality. Sometimes there can be great opportunities in the market that require speed to dominate the market but at a later stage, you can improve your codebase with refactoring. We see these patterns at start-ups, new technology-dominated products (Chat GPT integrations), or sudden market opportunities (q-commerce hype during covid).
Unfortunately, technical debts are only spoken by software engineering teams. It is almost impossible to hear these terms in data teams; e.g. data science, machine learning, data engineering, BI, etc. On the contrary, we have more severe technical debts in data teams because these teams’ priorities and initiatives are driven by stakeholders!
Some examples of technical debts at data teams;
- Data engineering teams receive new ETL/ETL requests every single day; e.g. adding new columns and tables to existing pipelines, and creating new pipelines. As a consequence more things start to fail in the middle of the night, delays happen at daily jobs, and stakeholders lose business-critical tasks because the data is not fresh enough.
- BI and analytics teams build dashboards, gather feedback from stakeholders, add new columns to the dashboards, add new charts, so and so forth. In the end, the dashboards become monsters with valid history and versioning. The innocent SQL queries become spaghetti codes that it is impossible to identify why the X table was added to the join or why the Y column is in this dashboard since it is not used.
Product teams
At successful product teams, engineering managers continuously observe and manage the technical debt problem in the team. Keeps the engineering team and product manager in the loop on the direction and the strategy on how to eliminate it. Technical debts are treated independently from the product initiatives and always have been placed during the sprints; e.g. 20% of each sprint is for technical debts.
Proactiveness and creativeness are just myths
Data teams
During the interviews, we ask data people to be proactive, self-starters, and have growth mindsets. We motivate them in the early days after the onboarding days and handover them a Jira backlog of 100 tasks.
When a data person takes over such a burden, how do we expect them to have enough time and space to be creative and proactive? Imagine yourself in this situation. You have open tickets that are open for months. There is no valid documentation about the past. No code base history and tickets linked to understanding why these exist at all.
In this scenario how do we expect them to deep dive into data and come up with insights to run the business?
Product teams
On the contrary product teams continuously run discovery sessions with the product manager, engineering manager, UX, design, etc, to find out new opportunities to lead the roadmap.
No direct impact and responsibility in company strategy
Data teams
In the majority of companies, data teams don’t have a dedicated seat at the senior leadership table. These teams are not represented by a Chief Data Officer or VP of Data. This problem may seem minor but it creates an avalanche effect on the whole organization.
The main problem with this minor seating problem is the voice of the data people is not heard while senior leadership designs the strategy and priorities, they don’t make data-informed decisions.
After the senior leadership starts the process, product, marketing, sales, and many other business teams make unrealistic assumptions and bets which are impossible to achieve because they were not evaluated in the first place and not backed by data. The rest is pretty easy to imagine; failing bets, teams start gossiping over each other, scapegoats are sought, and finally, a toxic culture is nurtured by the senior management team.
Product teams
Product teams own portions of the company strategy and they are empowered to lead that direction; e.g. if the company strategy is to increase the customer base by %10, the customer acquisition team takes ownership of the initiatives that can be driven by the product.
How should data teams be managed as Product Teams?
There is a pretty easy and straightforward solution to the root cause of these problems, and many more problems we didn’t deep dive into today; data teams should be led like product teams!
Shift from the project mindset to the product mindset
Projects tend to have a start and end date with predefined goals. When the project goals are achieved within or later than the end date, the project ends.
Data teams tend to treat the requests as projects. They assume when they deliver the requested subject, the stakeholders will not knock on their doors. The unpleasant reality is the opposite.
Let’s check some scenarios;
- If the request is successfully delivered on time, an extension to the report will follow up; e.g. adding a new column to the report.
- If the request is successfully delivered on time, another request will follow up quickly; e.g. first report checks X, Y, and Z for country A, in the second report checks Y, Z, and T for country B.
- If the request is successfully delivered on time, the results and/ or format don’t satisfy the stakeholder so a new request to fix the report is asked; e.g. bar chart replaces pie chart, last 3 days become last 7 days, etc.
We can add many more scenarios but the truth is that requests never end as projects with proper goals and timelines. If these are not projects, maybe data teams should start treating them as products. This requires a mindset shift and more details can be found in my “Product Thinking for Data Teams” article.
Define your data team’s vision, strategy, and roadmap
Product teams have a clear vision, strategy, and roadmap. These vision and strategy statements are also publicly shared in the organization and everyone knows what they are doing; e.g. acquisition team leads user acquisition areas in the organization and owners of X, Y, and Z KPIs, etc.
In the same way, data leaders should define, articulate, share, share, and share their team’s vision, strategy, and roadmap at every single time. This is the most important step to distinguish the data team from a service or support team and position it as a product team. More details can be found in my “How to be a Successful Data Leader” article.
Create competencies for the data team
In product and software engineering teams, leaders create competencies for their teams. These can be represented as matrixes to solve tons of problems;
- What are the skills needed to develop the product
- What kind of skills exist in the team
- What kind of skills missing to achieve the vision of the product
- How to create a team where everyone backs up each other’s skills
- How to create a successful hiring plan
- How to create a successful career progression and evaluation plan, etc.
At product teams there are different people from various expertise; e.g. backend engineers, frontend engineers, UX designers, product managers, etc. These people are gathered based on the requirements of the product to be developed.
When the vision and strategy are not there, it is impossible to evaluate which skills are needed at which levels. Therefore when the data team’s vision and strategy are set, the next thing is to identify which skills are needed to achieve the roadmap. Maybe a BI team will need UX support to build and educate users on how to use the dashboards or reports. Or a data science team will need an academic researcher while building a new algorithm that doesn’t exist. More details can be found in my “How to be a Successful Data Leader” article.
Define the impact of the data team
Every product team knows which metrics and KPIs they impact in the business; e.g. user acquisition team leads user acquisition-related metrics and KPIs, the growth product team leads growth metrics and KPIs, etc.
This is not different for the data teams. Data teams build data products that impact various business and efficiency metrics and KPIs; e.g. building automated reports and dashboards to improve the sales team lead generation process, developing early CLV machine learning models to improve customer acquisition and marketing metrics, etc.
When the impact is defined and shared within and outside of the team;
- Data team members become more aware of their impact. Their ownership and dedication increase.
- The data team is pulled to the strategic decision-making process since they have a direct impact on certain business KPIs.
- The executives become more aware of the data team and their existence. It is not seen as a support function in the organization.
Pull your technical team members from non-technical stakeholder meetings
Have you ever seen a backend engineer having weekly sync meetings with the stakeholders? Or a frontend engineer designing the product design and UX with the end users? Of course not! These people invest their time to develop the products. Then why do we push data analysts, data scientists, and business intelligence developers to meet with stakeholders to do these?
At product teams, there are product managers being a bridge between stakeholders, end-users, and engineering teams. These people collect the requests, define problems, discover opportunities, and convert them to actionable initiatives. Then meeting with engineering, product design, and UX teams on how to develop them.
In the data domain, we need data product managers to lead product management tasks. These people should be the bridge between stakeholders and data teams to develop the initiatives. Otherwise, we waste the energy, time, and motivation of our technical data people in non-technical and non-expert tasks.
Document your data products and train your stakeholders
Imagine the mobile applications you are using every day. Do you constantly get support to use the apps from the engineering teams? No! If you need support, you check the documents or how to use videos. If those are not helping, which may happen from time to time, you seek additional support from help centers. Your request turns into documentation and helps videos afterward to prevent recurring requests.
The product-led mindset ensures that the products are known and used by the end-users without the support of the developers. This should apply in the same way to the data products. While data teams build these products, they should generate extensive documentation and train their stakeholders. Otherwise, they face continuous questions from the stakeholders and cross-functional teams; e.g. missing KPI definitions at dashboards, non-clear REST API contracts from Machine Learning teams causing problems for Backend Engineers, etc.
Conclusion
The product-led development process is the most successful method in the last 10 years to lead the IT industry. We have learned a great deal of best practices from the product teams. In light of these learnings, it is time for a mindset change for the data teams. They should not be treated as support and service functions in organizations. Especially with the rise of Data Mesh, there is no alternative to migrating to leading data teams as product teams.
In this article, I tried to cover the core differences between data and product teams and shared some best practices that can be applied easily in every organization.
