Human-Centered Machine Learning

Jess Holbrook
Jul 10, 2017 · 13 min read

1. Don’t expect Machine learning to figure out what problems to solve

2. Ask yourself if ML will address the problem in a unique way

Gmail looks for phrases including words like “attachment” and “attached” to pop a reminder when you may have forgotten an attachment. Heuristics work great here. An ML system would most likely catch more potential mistakes but would be far more costly to build.

Describe the way a theoretical human “expert” might perform the task today.

If your human expert were to perform this task, how would you respond to them so they improved for the next time? Do this for all four phases of the confusion matrix.

If a human were to perform this task, what assumptions would the user want them to make?

Plot ideas in this 2x2. Have the team vote on which ideas would have the biggest user impact and which would be most enhanced by an ML solution.

3. Fake it with personal examples and wizards

Chat interfaces are one of the easiest experiences to test with a Wizard of Oz approach. Simply have a team mate ready on the other side of the chat to enter “answers” from the “AI.” (image from:

4. Weigh the costs of false positives and false negatives

The four states of a confusion matrix and what they likely mean for your users.

5. Plan for co-learning and adaptation

An example of the virtuous cycle is how Gboard continuously evolves to predict the user’s next word. The more someone uses the system’s recommendations, the better those recommendations get. Image from
The Google app asks every once in awhile if a particular card is useful right now to get feedback on its suggestions.
People can give feedback on Google Search Autocomplete including why predictions may be inappropriate.

6. Teach your algorithm using the right labels

Can you pass this quiz?

7. Extend your UX family, ML is a creative process

Work together with Engineering, Product, etc. to piece together the right experience.



Google Design

Stories by Googlers on the practice of design. For editorial content and more visit

Jess Holbrook

Written by

Co-lead of Google’s People + AI Research team (PAIR). Not surprisingly, I write about people + AI and the role of UX in AI-first companies.

Google Design

Stories by Googlers on the practice of design. For editorial content and more visit