Practical Guide for Product Managers Using AI
Product management is already a fairly ambiguous job that requires multiple skills: strategy, communication, influencing, execution, growth, data analysis to name a few.
There is one skill that is not discussed enough in product management and it is critical for all product managers to recognize early: use of AI.
Every company today uses some form of AI in their business stack as one of the following: (1) Growth Layer (2) Product Differentiator (3) AI-first product.
This does not mean that product managers (“PMs”) have to swap places with specialized ML engineers or head to learning all the tech courses on AI/ML. PMs need to proactively recognize the following:
I. There is a learning curve. Upskill yourself and your team.
The first academic paper on Artificial Intelligence can be traced to 1950s and invention of back propogation was back in 1980s. While these theoretical concepts have existed for a long time , these are still relatively “new” technologies in practice.
There is no shortcut for you to catchup without dedicating time and effort to it. Good news is that you are not alone and there is a plethora of online material for all levels of interests / skill levels to start with.
Your goal from upskilling should be to reach a point that everyone on your team speaks a shared language when it comes to these concepts.
Even though there will be a specialized ML engineer (or an entire team) responsible for the implementation, as their PM, you want to be able to articulate the user problems, business motivation and pros / cons of the proposed solution(s) in simple terms to other stakeholders in the team, leadership and to your clients.
Another thing to be aware of is that most often your use-case will be on the serving side i.e. your team will applying machine learning instead of designing it. There are ML focused teams that are designing TensorFlow or Pytorch libraries. If that is the case, you need way more hands on knowledge of Machine Learning.
II. Are you solving a real user need or force feeding a solution to your user problem.
Before your team makes the call to jump on the AI/ML wagon, check for the following:
Q1: Does this solve a real user need?
This should show up in your user research or competitive analysis. Be very clear about the user problem and what are the possible solutions before getting on the path to executing a specific AI-based solution that just sounds exciting.
Q2: Is there a non-AI / ML solution to your problem?
Spoiler Alert: Answer is always yes.
The caveat is that typically the quality of such a solution might be much worse through a rules based (“heuristics”) method. Most AI based solutions can be reverse engineered into an interpretable approach to some extent at least in the prototyping phase.
You can de-risk early by starting with heuristics and ensure that the team gets the building blocks like API, metrics and evaluation criteria right. The same pipeline will be needed for the future ML version as well.
Q3: Do you have sufficient data to actually model and what will it cost.
If you are planning to acquire (eg: user history) or purchase (eg: financial markets) the data, what are the costs associated with it.
Costs can be in the form of $$$, technical debt for your team, new product experience that puts burden of providing data on the user.
There are also recurring costs like (1) additional time / investment needed for data cleaning, (2) operational costs to maintain this data securely and in a usable manner and (3) meeting country specific regulatory requirements associated with this data.
III. Are you organizationally ready for the investment needed to train, test and launch your models at scale.
Now that you know your users can benefit from this solution and your team is excited about the possibilities, you have already realized that you cannot operate in a silo.
There will be other teams in your org that touch the product, effect the user experience and influence inputs to your team.
Consider the following carefully:
1: Where will the data sit, used and safeguarded.
Collecting , labeling, training data for any ML system can require extensive setup and maintenance. There would also be privacy , integrity, legal aspects to be aware of when it comes to using data.
It is a part of your job to continue to be the advocate for the users and push for high interpretability, integrity and transparency standards.
2: Do you have the right leadership buy-in and staffing to support this on an ongoing basis.
Any chosen solution will have tradeoffs across coverage, correctness and complexity. Create the right framework for your team to be able to decide across these axes.
3: Ensure you have diverse perspectives and are not leaving your other product partners behind.
As the team’s product manager, your confidence to carry forward the right solution for the users and for your organization heavily relies on having a seat for all relevant stakeholders: Design & Content, UX Research, Marketing along with Engineering and Data-Science.
Involving these relevant stakeholders early and having that shared language (as discussed above) to solve the identified user needs ensure that your team continues to have a fast iteration cycle and a successful product.
