Why Your Organization Needs a Machine Learning Product Manager
Machine learning is everywhere you look, affecting many technologies and products that we use on a daily basis. But who are the product managers leading these products? Who is ensuring that the success metrics are set correctly and ethically? Who is responsible for accurate messaging around such products?
Let’s go several years back, and look at the Product Manager’s role. This role has been around for a long while and is quite defined.
The product manager oversees the product’s ideation, design, and implementation; they analyze the user’s response to it and follow events and metrics when making subsequent decisions; they ensure that the product adheres to the company’s strategy and goals. In general, PMs are a channel of communication between tech, design, marketing, and business stakeholders. They are expected to know the ins and outs of their product and ensure its success.
Nowadays, we are starting to see more and more products involving Machine Learning algorithms and capabilities (think Spotify and even Netflix), so a PM’s role starts to take on a different, slightly more complicated angle. This is due to the highly technical (sometimes even unexplained) character of ML solutions, which raises the need for further exploration and investigation. Incorporating ML solutions into your product can influence anything from a product’s release timeline to ethical issues around it.
In my current role at Whisk, a Samsung Next product, I manage several products, some of which involve ML solutions (for example, recommendation systems). During this time, we have encountered many of the nuances I mentioned above. Our experience has led us to think extensively about the newly defined role of an ML PM that companies are going to need to integrate into their organizations — even if they do not know it yet.
In this post, I’ll share several insights about PMing ML products, in hopes of providing some clarity and guidance on how organizations (and individual PMs) can drive change and become better at this challenge!
Why you Need an ML PM
The technical questions that your PM asks change drastically once you have an ML component in your product.
Even in early stages such as competitor research, one needs to understand the AI solutions and trends out there. While the PM does not dictate the technical solution, they may want to understand for example why their competitors use BERT over GPT3 (two types of popular NLP language models). Did they do so for reasons of efficiency, coverage, or performance? What insights can we gain from our competitor’s approach towards ML solutions?
In stages of specing the product requirements, an ML PM should understand from their Data Science / Algorithm team how their choice of solutions affect timelines of development, as well as if the chosen solution can be scaled as needed, and can the model be re-trained easily. Naturally, they should also consider whether or not this solution will be easy for users to understand. Obviously, this is just the beginning and there are so many more questions to be asked.
Product managers are usually responsible for defining a product’s goals and OKRs, which requires them to have an understanding of ML algorithm success metrics. For example, when building a classification algorithm, an ML PM should define what would constitute “good performance”. Product-wise, is precision more important than recall? Is accuracy enough? Are all types of predictions equally important, and how do these definitions align with the company’s strategy?
These are just several examples of technical questions requiring that your PM “speaks ML” fluently.
Another sensitive issue is data. The PM who leads ML-based solutions should be fluent in your organization’s data and what can be done with it. Will there be a need to label data for this solution, and what are timelines repercussions? Also, to ensure that users’ expectations are met, the PM must know if the data is representative enough. Unfortunately, there are too many cases of ML solutions being trained on biased data, which may affect large populations. With ML solutions touching upon people’s credit score, medical state and actual future, biased data is no minor issue and an ML PM should play a major role here.
Communicating ML to Non-Tech Stakeholders
As previously mentioned, a PM is often a funnel of communication between tech, design, business, and marketing. Some of these stakeholders (and, of course, our users) may be non-technical, and won’t understand what you want with “all that AI stuff”. Therefore, a PM leading an ML solution should feel comfortable communicating ML concepts to non-technical stakeholders. For PMs interested in this challenge, my recommendation is to learn to simplify ML and communicate it regularly! Write blogs about ML in simple language, present in meetups, teach courses, etc.
This challenge will appear constantly. When presenting monetization models for ML solutions to your business peers, when putting together marketing materials about your recommendation system or facial-recognition solution. It will come up in user interviews, when users want to know what is being done with their data, and will come up when you explain your decisions to management.
But isn’t ML a Research Field?
All PMs deal with conflicts of interests, and quite frequently so. Designers will push for a perfect UX, developers will push for simplicity and efficiency, and management will push for stricter timelines.
The same is true in the ML domain. Data scientist / ML engineers will often be interested in state-of-the-art solutions, and as a former data scientist, I fully understand them! However, the ML PM must remember that there’s not always the time, resources (data), or need for such extensive solutions. Their mission is to make sure that the team is aligned on applicable ML solutions vs. research/experimental ML solutions (unless, of course, this is a research team).
My tip to ML PMs here is never dictate a solution to your tech team (be it a simpler solution or a state-of-the-art one). They are the tech experts. However, always raise all important concerns and focus on the user’s needs. Lead in the direction of applicable, realistic, and suitable ML solutions (even if they end up as heuristics, and not proper ML).
So how can we Hire a Successful ML PM?
Once organizations understand the importance of bringing on an ML-oriented PM, what are the qualities and characteristics that they should search for?
First and foremost, I would recommend someone that has actual experience with Machine Learning and associated technologies, preferably hands-on. For example, consider former data scientists, algorithms engineers, or people of similar backgrounds (don’t forget, having a background in ML doesn’t necessarily mean that candidates must have a PhD).
To ensure success, make sure that you bring on good communicators. While this is true about any kind of PM, I really want to drive this point home here due to the complexities that come along with ML product development. Your ML PM needs to be able to communicate — in simple terms — decisions, requirements, and the general roadmap to all segments of the company, from R&D to Marketing and Sales, just like the traditional PM would do. One suggestion I have is to incorporate a “communicate ML to non-tech” challenge in your interview process, to make sure that your candidate has this down!
Make sure that your PM has a firm grasp of how to market ML products. Figure out if they know how to phrase the correct messaging for such ML solutions. Additionally, they should have the visibility to think about how a specific ML feature can fit into other products and the “AI Market” in general.
Final Thoughts
Hiring the PM who understands the complexities of working with ML and knows the right questions to ask can reduce friction and accelerate the path of getting products to market in a reasonable timeline. A strong communicator with an ML background can both educate others in the company, and strengthen your AI-value prop and messaging.
As with any role, recognizing the need for an ML PMs is the first step in moving in the right direction for any company looking to incorporate ML into their products. I’m proud to be working with a team that has internalized this new reality and is taking measures to implement it into how we think about product management!
