3 Common Problems With Your Machine Learning Product and How to Fix Them

Clemens Mewald
Nov 30, 2018 · 7 min read

Unlike traditional products, the launch of ML/AI-driven features or products is just the start of a Product Manager’s role. ML models introduce a dependency on data (see our previous post), as well as some level of uncertainty in outcomes. In the real world, this can lead to a number of undesirable outcomes that a Product Manager needs to anticipate and mitigate. In this post, we will discuss how to protect your ML/AI product from time, from adversaries, and from itself.

Protect your ML/AI product from time

Image for post
Image for post
Figure 1: If you try to use ML to give fashion advice, know that fashion changes over time (Image source 1, 2)

The fact that ML learns patterns from data should give a Product Manager pause and lead to the logical follow-up question: What if those patterns change? We call this general problem “model staleness”, which refers to the predictive power of an ML model decreasing over time, as trends or tastes change. Take for example a product that suggests fashion items to shoppers. Due to the short-lived nature of fashion, the product’s suggestions may become irrelevant quickly (although, if you wait long enough, any fashion trend may have an unexpected revival).

To remedy model staleness, you will have to refresh your model over time to allow it to learn from new data. Below is a graph that shows how you can avoid model staleness over time by refreshing the model with new data periodically.

Image for post
Image for post

However, there are two important considerations when going through a model refresh: 1) Not all ML/AI products are susceptible to staleness in the same way, and 2) refreshing models can introduce unwanted feedback loops.

  1. How worried you have to be about refreshing your models depends on how “stationary” your ML problem is. A product that distinguishes cats from dogs is relatively immune to model staleness. The evolution of cats and dogs is not fast enough to render your model useless over the next few years. However, if you are building a product that recommends content to your users (like videos, or fashion items), or one that detects fraud, these types of problems change rapidly and ML models need to adapt.

Protect your ML/AI product from adversaries

Adversarial actors, like their attacks, come in all shapes and sizes. To broaden the list of concerns, bad intent is not always necessary to induce a negative effect on your ML product. Three high-level categories of adversarial behavior to consider are theft of your ML models, taking advantage of feedback loops, and coercing your ML models into bad behavior.

Image for post
Image for post
(Image source)

Theft of your ML models

If you expose your ML model directly to users (e.g. a model that allows users to predict housing prices by entering information about a specific property), there is a way for adversaries to steal the ML model through observing input and output combinations.

Most products do not directly expose their ML models to their users, but if you think there’s a chance, have your engineering team start with this paper and think carefully about protections. One way of protecting against such attacks is rate limiting (i.e. not allowing a single user to make enough prediction requests to steal a model).

Taking advantage of feedback loops

The most common and potent way to influence an ML model’s behavior is to take advantage of feedback loops. Above I described how feedback loops have to be taken into account when refreshing models. They also have the potential to allow adversaries to negatively affect the user experience by tilting the balance in their own favor.

Returning to the video recommendation example, if your ML model recommended videos based on how frequently they are watched after another video, an adversary could bias the model in their favor by compensating many people to navigate to their video directly from other popular videos.

Solving the “false traffic feedback loop” problem requires even more ML: Work with your fraud team to differentiate fraudulent or spammy clicks from “authentic” clicks, and block them (or, at least, don’t include them in your training data).

Coercing your ML models into bad behavior

A well-publicized example of ML-gone-wrong is a chatbot that was trained through online conversations and learned from its interactions once it was launched. That bot learned that profanity was a common way to express feelings, and to more generally parrot the language and points of views of its conversation partners.

The lesson for Product Managers is simple: Your ML models will learn from data, if the data are bad, your models will be bad. As a result, you should always put safeguards in place. The general concept we use for such safeguards is a “policy layer” that sits on top of an ML model (or system) and enforces a well-defined policy. Examples include:

  • Prevent reinforcing bad habits: Following the chatbot-gone-wrong example from above, it is important to identify and prevent bad habits (e.g. profanity and gender pronoun biases) that may be apparent in the training data. Adversaries can intentionally coerce an ML model into bad habits by providing bad training examples. One way of dealing with this is to run any output of an ML model through a policy layer, like a “bad language detector” (yes, more ML!), before showing it to users.

Protect your ML/AI product from itself

ML models blindly learn to optimize a given metric based on the data they observe during training. By now it should be well known that, if those data are flawed, the models will be flawed. As a Product Manager, your job to safeguard your users from mistakes and biased predictions.

  • Error analysis: Before launching any product powered by ML/AI, you need to spend considerable time doing error analysis. Note that this is different from the technical errors described above. Error analysis refers to instances where a model makes a prediction that is wrong or unexpected, like a false positive. In short, you should systematically investigate the cases where your model makes the worst predictions and either take steps to improve your model or to fail gracefully.

Conclusion

I hope I have successfully convinced you that the ML/AI product launch is only the beginning of the PM’s job. The update cycle of traditional products is driven by user expectations for new features and software release cycles. The added dependency on data and the probabilistic nature of ML/AI predictions lead to a much higher-frequency cycle of updating incremental learning systems. In theory, every interaction with a user provides an opportunity to update the ML model, impacting your product.

These specific challenges need to be addressed by Product Managers, including protecting models from going stale, anticipating adversarial actors, and making sure that ML models themselves keep behaving the way they should. Stay tuned for future posts!

Clemens Mewald is a Product Lead on the Machine Learning X and TensorFlow X teams at Google. He is passionate about making Machine Learning available to everyone. He is also a Google Developers Launchpad mentor.

The Launchpad

The Launchpad is a resource for applied-Machine Learning…

Clemens Mewald

Written by

I lead product for Data Science and ML at Databricks. Previously I was a PM lead on the Google Brain team, building AI infrastructure for Alphabet.

The Launchpad

The Launchpad is a resource for applied-Machine Learning (ML) best practices from the trenches. Our goal is to help AI-first companies innovate & thrive, sharing knowledge & insights from our community:startups, mentors & Googlers https://developers.google.com/programs/launchpad/

Clemens Mewald

Written by

I lead product for Data Science and ML at Databricks. Previously I was a PM lead on the Google Brain team, building AI infrastructure for Alphabet.

The Launchpad

The Launchpad is a resource for applied-Machine Learning (ML) best practices from the trenches. Our goal is to help AI-first companies innovate & thrive, sharing knowledge & insights from our community:startups, mentors & Googlers https://developers.google.com/programs/launchpad/

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store