AI-Based Product Development at Zillow
AI-powered technology is a new way to design products. Farah Abdallah Akoum, Product Manager at Zillow, shared her insights on what to focus on while developing AI products at our Product People Community event.
Artificial Intelligence is present in our daily life ranging from Siri/Alexa to iRobot cleaners to Stock market predictors. Being a Product Manager of AI Products not only requires understanding the basics of AI but also the differences in the AI Product Lifecycle as compared to the traditional Product Lifecycle.
“ AI is augmenting human capability and capacity. Its defining technology of our time” — Satya Nadella
Enabling businesses to solve customer problems by adopting the right principles when developing AI Products is one of the key responsibilities of a PM. Every technological revolution forces the Product Managers to rethink business and customer needs. In the last decade, one had to rethink the design of user flow in small screens with mobile and well as the value proportions in mobile like location, notifications. Similarly, including AI in products also requires some re-modification in traditional product development methodologies.
Difference Between AI and Traditional Product Lifecycle
- User Research & Design
▹ Traditional Product Lifecycle: Starts with user research and understanding the customer problem and making low fidelity designs.
▹ AI Product Lifecycle: This also starts with customer problems without focussing on the AI technology initially, however, to understand the customer problem data discovery and analysis is performed and the measurable objective is defined. - Feature Definitions
▹ Traditional Product Life Cycle: Focus on the creation of a product requirement document (PRD) listing the user features and the steps to be performed by the user and the prioritization of the features.
▹ AI Product Lifecycle: In AI products, the focus is on the constraints and features of the AI model rather than the user features. Understanding the variables to be defined and the outcome of the model is the crucial steps for AI products. - Relationship with the Development Team
▹ Traditional Product Lifecycle: PM’s role is to partner with dev teams and ensure that the solution is scalable.
▹ AI Product Lifecycle: In AI products, the role of a PM is to ensure that the data prepared by the data scientists are unbiased and the correct algorithm is considered in building the product. - Product Launch
▹ Traditional Product Lifecycle: PM collects the data to decide the next feature which we want for our users.
▹ AI Product Lifecycle: In AI products, we try to teach the model how to learn more about the customers by recognizing the errors it makes.
AI Product Lifecycle
Step 1. Define an ML Problem and Propose a solution
A measurable objective is the first step in the creation of the AI Product. For instance, at Zillow, the vision is “Customers find their dream home effortlessly.”
Zillow is an AI-based application which recommends users the value of the house for selling, renting or buying.
Initially, as pointed out by Farah, there were 200 attributes to narrow the search. However, during data discovery, they found out that the customers who clicked on contextual filters were more likely to contact the agents. Hence, the objective was to provide the customers with the contextual filters to optimize the process of finding the house for which they were willing to contact the agents at Zillow. Setting the right objective also helps to narrow down the relevant metrics for the product.
Once the objective is defined, finding the right algorithm, the input and the output for the ML algorithm is the next crucial step. For a PM, it is important to have basic knowledge of ML algorithms and their use cases in real life.
Supervised learning uses labeled datasets to predict the relationship between the input feature and the output target. Unsupervised learning is one that has no labels to the data.
Step 2. Construct and Transform the Dataset
To build the AI product the major requirement is to prepare the dataset without which one cannot think of solving the problem using AI.
Having a perfect dataset is the key to a successful AI product. The main focus should be on three important points:
1. Data collection: Data related to user activity, both positive and negative user feedback, and data provided by 3rd party needs to be collected.
How do we collect data?
One of the easiest ways is to find an already created dataset, as Google offers a dataset search engine. Other ways could be to get a dataset and perturb data and inject different variations in the dataset, crowdsourcing using Amazon’s Mechanical Turk, or the intensive work of collecting data manually.
2. Data Quality: Once the data is collected the next concern of a PM should be the permissions for accessing the data, data fairness, and transparency. The legal and ethical considerations are to be kept in mind.
For example, in the case of medical datasets, there are many policies that need to be considered such as HIPAA (Health Insurance Portability and Accountability Act of 1996), GINA (Genetic Information Non-Discrimination Act) — you can learn more about this in the course on AI Product Management from Stanford by Ronjon Nag.
3. Data Preprocessing: After the data is collected and checked for quality and all the legal restrictions, the developers clean the data so that the noise in the dataset is removed.
Step 3. Deploy the Model
In the case of AI products, one cannot control the outcome of the model. The major goal is to train the model to reach the objective. The success of an AI product is based on the scores such as accuracy, precision, and recall produced by the model on the validation dataset. Also, if possible the model should be tested offline and once all the integrations of the frontend are ready it could be tested online with the in-context user feedback. Besides, the model should be iteratively tuned and A/B tests should be performed to achieve the desired results.
AI-based Product development will lead to reinvent and rethink the way we develop software. Starting from analyzing and understanding the user needs followed by thinking of ways to include AI as a solution for those needs. Creating innovations and great user experiences is the goal of AI-powered products.
Watch the full version of Farah’s talk for more insights and a great Q&A session.
Subscribe to our YouTube channel. And join the next Product People meetup for more inspiring stories from Product Managers from around the world!
