Tomorrow’s Product Managers Will Need Solid Data, Model and Problem Understanding

When people talk about Product Management of the future, the first theme that comes to mind is artificial intelligence (AI). AI is changing the fundamental structure of every industry. We’re interacting with technology in new ways, from giving voice commands to virtual assistants to having Smart Reply suggest quick responses to our messages. In Alpha’s Product Management 2018 Insights study, a third of respondents reported having AI or machine learning incorporated into their products.

Does that mean it’s time to embrace the mathematical techniques that enable the building of intelligent software applications using the family of techniques known as deep learning? Absolutely! There’s no doubt that delegating parts of the software development process to machine learning models is going to be mandatory.

As we have become accustomed to having customer research and User Experience (UX) lead product vision, a new problem is stopping eager Product Managers from putting the process they were using, and applying them on AI problems. The reality is that we will need to evolve by finding ways to have a solid data set understanding. Product Managers can start implementing AI by adopting a bottom-up approach. First, create/collect a data set that represents the problem space. Next, collaborate with data scientists to understand the abstraction of that data set.

The Current Product Management Process

Not all products are created equal. Where there is no deep learning the process becomes sequential. Product Managers gather real-world feedback and make decisions on what features to build. They lay out very clearly what they have learned about the actual customers, through qualitative or quantitative knowledge. State the additional problems they look forward to solving and propose solutions to grow the customer value while tying them to business results. They make a pitch, and explain why it’s worthwhile for the company to invest in the solution. They then step back and trust the team to develop the prioritized features. Designers come up with mockups and engineers building the features jump in. While Product Managers inspire through vision, decisions roll downstream and tend to be more implementation based. Like an assembly line.

ML systems are trained on existing data sets.

Machine learning (ML) on the other hand is a complex space where results are achieved through frequent iterations. Through a large dataset of similar cases the computers discovers patterns and relationships in data instead of being manually programmed.

“It’s software writing software.”

Where there is machine learning the process becomes everything-is-just-a-probability. Faced with no-exact-numbers-and-definite-results Product Managers get a little clueless about what they should focus on. Algorithmization (dependency on deep learning (1)) requires strategic data acquisition, from the onset. Most issues with machine learning are solved by understanding and preparing data better, and that’s the first responsibility of the Product Manager. AI engineering teams benefit from interpretable and understandable solutions, so the Product Manager must assemble a small, diverse team to tackle the issue. They become involved or involve Data Engineering and Data Science which extracts meaning from and interpret the unique composition of the data set.

Deep learning has become the go-to technique for most AI type problems

Deep learning forces Product Managers to focus more on the data to train the software. They relinquish much of their dependence on User Experience and real-world feedback adopting a trial-a-error iterative process meant to prevent skewed outputs. For example the Gmail platform predicts the quick responses according to the message received — focusing time conducting user research and tinkering with wireframes won’t help the team understand where the biggest improvements need to be made.

Product Managers need to ensure that data scientists are delivering results in ways business counterparts can understand, interpret, and use to learn from. This includes everything from the definition of the problem, to the coverage and quality of the data set, its analysis, to the presentation of results and follow-up. While keeping the team focused they collaborate and brainstorm together with Data Science and Engineering to clarify what to do with the information collected. The nature of ML and AI forces teams to work collectively since the risks are far greater. It gets them to communicate better and a common, shared understanding of the end goal develops organically.

It’s a bottom-up process, you attempt to solve a problem with the team, get a signal early and use it to construct the bigger picture. By knowing your training dataset cold you set the vision and direction of the product.

As I discuss in my article: WatchMojo’s immersion into Deep and Machine Learning, being able to construct summaries and visualizations that examine why the model works illustrate not only the importance of validation to improve a product but that teams always prefer being led by someone who invested in a baseline to understand the data.

From Analytics to AI

We’re closer than ever to creating some of the most innovative and smart AI technologies the world has ever seen. Below is a roundup of advice on how to understand the abstraction of data.

• Get the right data set: Look at your problem space and collect the unique data source first. Label data and generate a stronger dataset. Then constantly look for additional sources of data that will continue to improve your product(2).
• Opportunity assessment:You’ll need to first understand where the biggest improvements can be made. The most important element of data gathering is what to do with the information once you collect it(3).
• Focus on eliminating bias: Besides collecting the right data, another big step is ensuring that the data that you work with is correct. Bias is potentially present in any dataset and it is up to you to be aware of potential bias and work with a data scientist to resolve it.
• Weigh the cost of getting it wrong: Cover all scenarios that the Machine Learning Module (ML) might have to encounter. It’s important to understand what errors look like and how they might affect the user’s experience of the product.
• Know how you’ll know if you are successful: Have a solid problem understanding. Link and correlate outputs back to the input data. Outputs should correspond to your intuition. Compare how many times the ML gets it wrong in comparison to a human.
• Build a safety net as well as a feedback loop: Safety nets(4) are visible to users (UX) and take into account that erroneous actions can have consequences. Feedback loops allow you to monitor prediction accuracy, etc…

When what to do with the information collected becomes clear, engineers can figure out what tools and models are necessary to solve it. Only then will they iterate and fine-tune models to deliver accuracy. They will seek optimal performance because the product manager created model interpretability — that ability to verify that what the model is doing is in line with what he expects.

—

I am a CEO, Product Advisor and Founder who has experienced different levels of product management positions and has learning’s to share. I thrive in middle market and scrappy startup environments, where everyone does a little of everything. I work closely with CSuite leadership teams towards product-market fit and operating at scale. Check me out on BainPublic.com and view my LinkedIn profile.

Thanks to Sydney Swaine-Simon, Andy Mauro and Nis Frome for reading drafts of this. Also, if you have any feedback or criticism about this article then shoot me an email.

Notes

• (1) A Glossary of Artificial-Intelligence Terms By Roger Parloff in Why Deep Learning Is Suddenly Changing Your Life
• (2) Jeremy Barnes has a fantastic article he wrote on the AI-First business model. In it, he mentions the cost of acquisition of data and whether it’s better to own your own or acquire it from a third party.
• (3) Emmanuel Ameisen, recommend in his article “Always start with a stupid model, no exceptions.” that in machine learning like in most fields, it is always valuable to start with the basics.
• (4) Josh Lovejoy and Jess Holbrook provide 7 steps to stay focused on the user when designing with ML.
• Follow up readings specifically for the 6 points listed. Toward ethical, transparent and fair AI/ML: a critical reading list for engineers, designers, and policy makers

Notes

• (1) A Glossary of Artificial-Intelligence Terms By Roger Parloff in Why Deep Learning Is Suddenly Changing Your Life
• (2) Jeremy Barnes has a fantastic article he wrote on the AI-First business model. In it, he mentions the cost of acquisition of data and whether it’s better to own your own or acquire it from a third party.
• (3) Emmanuel Ameisen, recommend in his article “Always start with a stupid model, no exceptions.” that in machine learning like in most fields, it is always valuable to start with the basics.
• (4) Josh Lovejoy and Jess Holbrook provide 7 steps to stay focused on the user when designing with ML.
• Follow up readings specifically for the 6 points listed. Toward ethical, transparent and fair AI/ML: a critical reading list for engineers, designers, and policy makers