Data Product Manager: The Most Crucial Job of the 21st Century

Published in

Data And Beyond

7 min readJan 16, 2023

Eleven years ago an article was published, “Data Scientist: Sexiest Job of the 21st Century.” It shook the world from the ground. Everyone suddenly started to talk about this mysterious group of people. I remember some of the questions on social media platforms; are these people human beings, are they good at everything about IT, is it true that without a Ph.D. it is impossible to be a data scientist, are they going to change the world, did you hear that machine learning is going to change everything?

Before that article kids were aspiring to be firefighters, astronauts, and doctors, and suddenly every kid wanted to be a data scientist. Maybe not the kids but parents for sure! Due to the great marketing efforts, high demand was populated. Of course, supply needed to be created; online courses and universities started to teach data science. People began to make investments in those courses and academies. Data science hype created great opportunities also for data engineering, machine learning engineering, business intelligence engineering, etc. Suddenly it became cool to be a data person!

After the hype in the data domain, we faced the reality;

85% of big data projects fail (Gartner, 2017)
87% of data science projects never make it to production (VentureBeat, 2019)
“Through 2022, only 20% of analytic insights will deliver business outcomes” (Gartner, 2019)

Today I am not going to write about data scientists or I am not going to start an exhibition on finding the sexiest job of the century. Today I am going to write about a role that has not been publicly spoken of and undervalued for a very long time. This article is about “Data Product Manager: The Most Crucial Job of the 21st Century”.

Who should read this article?

Chief Product Officers (CPO) only focused on the core software engineering products, and never focused on the data products.
Chief Data Officer (CDO) hired brilliant data people and convinced the senior leadership of the investments but struggled to go beyond the “cutting-edge” data infrastructure and deal with data employee churn every month.
Data Engineering, Data Operation, Data Analytics, and many other amazing data teams built amazing data pipelines, data catalogs, dashboards, and many other items. Still, it was undervalued or even punished when two different dashboards showed different numbers for the same KPIs.
Data Science and Machine Learning teams came up with out-of-the-box ideas and solutions to solve customer problems but never managed to convince stakeholders to go beyond the Jupyter Notebooks.
Product Managers who are interested in data products but don’t have a clear vision about the future.

We create our own monsters

In the data community, there is a tendency to build data science, data engineering, and machine learning engineering teams without proper planning on what is needed from the organization. Then we throw the technical data people into the ocean of business and expect them to collaborate with the stakeholders, discover and prioritize all the initiatives, develop their machine-learning models, work with software engineering teams to integrate the models into core products, etc.

This is more like a fairy tale! We create our own monsters and expect the impossible for the data people. If this system worked, then every single software engineering team wouldn’t need a product manager.

Who is a Data Product Manager?

Data products are developed by data teams and they require data product managers. 2 + 2 = 4. But who are they?

I tried to use Wikipedia for the data product manager definition but it looks like it doesn’t exist.

When I ask the same question on Linkedin I have got 2.4 million people with this search result.

This is a quite dilemma. You can almost find any information on Wikipedia but the job definition of 2.4 million individuals is missing.

Let me share my description for this niche role is;

A data product manager is responsible for the development of data products for an organization. Data product managers own the data product strategy behind a product or data product, specify its functional requirements, and collaborate with data teams and software engineering teams to plan the data product delivery process.

What is needed to be Data Product Manager?

Data product managers are technical product managers in the data domain. They work with data science, machine learning, and data engineering teams to build data products.

On top of the core soft skills of product managers, they require several technical skills;

Data creation and consumption lifecycle

Without the data, a data product can’t be developed. A data product manager needs to understand the fundamentals of the data lifecycle;

How internal data is generated
How external data is collected
What are the data quality and validation operations on the data
How data flows between upstream and downstream systems
Who produces and who consumes data
What are the data contracts, etc

Theoretical knowledge

Today the majority of the data products are data science and machine learning products. They are developed by data science and machine learning teams who are experts in statistics and mathematics. A data product manager needs a great amount of theoretical knowledge in mathematics and statistics to understand the theoretical and applied nature of data science and machine learning models to manage these products.

SQL

Every organization one way or another store the data; e.g. data warehouse, data lakes, data lake houses, etc. A data product manager needs to access the data points, query whatever is needed, and gather the required information independently. Especially on early-stage start-ups or busy data teams, it can be a really big challenge for product managers to find someone to support them with data-gathering topics. SQL is a lifesaver for these types of situations!

Who should consider being a Data Product Manager?

Technical Product Managers

A technical product manager is a product manager with a strong technical background that is typically focused on the more technical aspects of the product. Technical product managers usually come from an engineering background.

Since these people have already a technical background and also the tendency to work on the technical side of the product, learning the data domain and its infrastructure will be easier compared to product managers.

Data Scientists

Data scientists are already great at the theoretical and applied nature of data science and machine learning topics. They are the ones expected to build the data products. The only challenging part for data scientists to be data product managers is the product management part. Because up to now they usually worked on projects; e.g. Kaggle competitions, online course-ending projects, and proof of concepts.

While projects have deadlines and ending goals, the products are continuous objects and they don’t end. Getting used to this mindset, improving stakeholder management skills, and keeping the Opportunity Solution Trees (OST) updated all the time, etc are the core product management skills data scientists need to learn.

Are companies prepared for the Data Product Managers?

This is a niche role and the companies are not prepared to evaluate, hire and retain these people in the organizations. The ratio of the data product manager roles in the companies is too low compared to how people identify their jobs.

In most situations, data product managers are hired and retained as product managers but are asked to do data product management. When it comes to evaluating these people, we hit walls because the product manager competency matrixes are not optimized to evaluate data product managers. We can easily find ourselves evaluating data product managers with their UX and UI skills!

Conclusion

Today we are surrounded by thousands of data products. These products are different from software engineering products. They require specific teams to build and manage them. Data product managers are the core of these teams. They are the bridges between the data producers and data consumers. Without their expertise, we are going to repeat the vicious cycle of failed data projects. In order to break this cycle, organizations need to change their thoughts about this niche role and get prepared how to find, evaluate, hire, and retain them. Otherwise only a small percentage of the companies are going to embrace the change and make a difference in the competition!