Designing with AI
What I learned from designing an artificial intelligence–enabled experience
At Facebook, AI is everywhere.
Behind the scenes, AI helps make Facebook smarter and easier to use. We use it to help translate text so people can understand each other better, to recognize what’s in images so visually impaired people can “see” the photos their friends post, and to filter out undesirable content like spam. We also use AI to understand the intent behind what people post so we can improve their experience on Facebook.
When I started as a designer at Facebook, I hadn’t thought much about AI or how it could be used as a tool in product design. But then I ended up designing Facebook Recommendations, which uses AI to detect when people are asking for local recommendations, and then to match the places that their friends recommend to Facebook Pages. It’s one way we help connect people and local businesses.
It All Started with a Problem
I’d noticed a lot of posts in my Feed in which people were turning to Facebook to find recommendations for places to go and things to do. These posts got a lot of engagement, but weren’t very useful. You had to scroll through all the comments, then copy and paste the names into Yelp or Google to find out more about the places your friends were recommending — even though over 60 million businesses have Pages on Facebook. The worst part was that it was really easy to lose these posts in your Feed, so they were only useful for as long as you could find them on your Timeline.
I wanted to make it easier and faster to collect and consume the recommendations that people get from their friends — and help them get more recommendations from people they trust.
How We Arrived at AI
People were already asking for recommendations on Facebook, and we didn’t want to get in the way of the behavior that was already happening. Ultimately, AI turned out to be the best approach because it let us make their posts better by turning their unstructured conversations into a helpful shortlist or travel guide. We didn’t reach this solution right away, though. First, I explored a number of possible solutions to our problem, which we validated through user research and live experiments.
One of the first concepts we tested was an approach where you’d have to say up front what you were looking for and where. But we found that people didn’t really understand why they would do this. They didn’t see the value of adding this additional metadata to their post, and it was tough for us to demonstrate the value we could give them when they couldn’t actually see the experience for themselves. We were also fighting against existing behavior, which was to just write a status update with their question.
Another concept we tested was a more educational approach. We thought that by stepping people through a tutorial that explained what was going to happen, we could help people feel more comfortable with the product. We found, again, that it was difficult to explain in words or illustrations how we were going to provide value before letting them experience it for themselves. Once people used the product for themselves, they loved it, but not surprisingly, we saw some drop-off in usage when we added in additional steps before posting.
From testing these more structured approaches, we learned that the less friction we added to the experience, the better it was for people. We decided that the best approach was something “automagical” that would augment the behavior that was already happening, without being too intrusive. In order to trigger the experience in a frictionless way, we relied on artificial intelligence to understand when people were asking for recommendations and what places their friends were recommending when they replied.
How Recommendations Works
With Recommendations, you can post your question on Facebook as you normally would, and when a friend comments with a recommendation, we link to the corresponding Facebook Page and display details like ratings, price range, open hours, and addresses. We also put all the places they recommend on a map so you can find everything easily. We’ve seen people use Recommendations to find everything from water during a hurricane in Florida to the best craft breweries in Australia. There are even Facebook Groups, such as the Tri-State Restaurant Club, where almost every post is a request or offer for Recommendations.
The AI Behind the Product
The Recommendations product seems pretty simple. Making it work is a lot more complex. In order to turn a status update like “Friends! Where’s the best place to get a haircut in Chicago?” into a Recommendations post, we have to first understand: (1) that you’re asking for recommendations, (2) what types of places you’re asking for, and (3) where you’re looking. This is easier said than done, especially considering the way people use slang and casual language on Facebook.
We partnered closely with the Conversational Understanding team at Facebook to use Natural Language Understanding (NLU) to power our experience. This team built AI technology that can understand text posts to accurately detect when someone is asking for a local recommendation, enabling us to automatically trigger the Recommendations experience.
When your friends comment on your post, suggesting all the cool places you should check out, we use AI to understand the text and extract the most likely place(s). AI also gives us a confidence score that indicates the likelihood that it’s the right place. This score shapes the user experience that the commenter receives. If it’s high, we simply attach a place card to their comment (with the ability to remove it). If we have medium confidence, we ask if it’s the right spot before attaching it. When the score is low, we show them an empty card that opens up a search bar that lets them manually search for the place that they want to add.
What I Learned About Designing with AI
The allure of AI is that it can make your product “magically” work. But my experience on Recommendations hammered home that AI’s power doesn’t lessen the need for thoughtful product design — just the opposite, in fact. Of all the lessons from the project, here are the ones I keep coming back to:
Look for existing behavior
AI opens up a lot of opportunities to make existing behaviors faster and easier for people. We didn’t try to invent a completely new behavior; rather, we found an existing one and made it way better! AI made it possible for us to deliver a magical experience while introducing as little friction as possible for people to give or receive recommendations.
If you don’t notice the AI, you’re doing it right
When you use AI in a way that enhances an experience, rather than defining it, it can actually feel almost invisible. AI lets you break free from a traditional user interface and solve problems for people in a seamless and almost magical way.
We purposely decided not to make it feel like you were talking to a robot or like Facebook was interjecting itself into a conversation you were trying to have with your friends. Instead, we used a design that augments the comments you receive from friends with helpful information. This approach feels more natural, and keeps your conversation with your friends at the forefront. In fact, in user research, when we showed the experience to people who said they’d never seen it before, some said “Oh yeah, I’ve used this before! It’s great!”
Test, test, test
As soon as you start introducing “magic” into the experience, people assume that it should work all the time. When we first started testing our experience, our AI got things wrong. A lot. It’s not a great experience when we suggest that you link to a dentist’s Page when you’re trying to recommend a restaurant. Usability testing with real people was very important, especially once we had a working product. We also looked at a lot of public Recommendations posts to get a feel for how well the product was actually meeting people’s needs. By watching people go through our experience early, we uncovered a lot of issues with our AI we wouldn’t have noticed otherwise.
Don’t depend on perfection
Even if your AI works most of the time, there are going to be moments when it completely fails. If people can’t accomplish their goal when this happens, you’re going to end up with a lot of very unhappy users. One thing that makes Recommendations work is that even if our AI didn’t recognize your post, it didn’t stop you from posting your question and getting recommendations from your friends — your post just wouldn’t be as helpful.
Degrade gracefully
By falling back to alternative UI, you ensure that you can provide a good experience for people even when your AI fails. Though it was more challenging to design multiple UI treatments, we were able to provide a less intrusive product by differentiating the experience based on AI confidence. For Recommendations, we experimented with raising and lowering the confidence levels and with a number of different UI treatments at each level, until we found the combination that worked the best.
Feedback is a gift
This is a saying at Facebook, and it’s especially true when designing with AI. It’s important to provide ways for people to give feedback about our guesses and then use it to improve the experience. By letting people x-out our suggestions when we were wrong, we not only provided an escape hatch, we also created a way to collect valuable information about how our AI is doing. Every piece of feedback we receive helps to improve our AI and the overall Recommendations experience.
Influencing AI’s Future
Since launching Recommendations, we’ve continued to improve the experience and I’ve continued to learn a lot about the challenges of AI and the opportunities that it can unlock. Above all, I’ve learned that designing with AI is just like designing anything else. Focus on people problems, test your assumptions, and provide affordances for when things go wrong.
As AI inevitably becomes deeply integrated with the products we build, it becomes more and more essential for product designers to take part in how it evolves. It’s a natural fit — making technology feel human was obsessing product designers long before AI came along. By applying design thinking to AI-enabled products, we can help ensure that those tools truly serve the people who use them.
