Search, recommendations and a fault tolerant UX

How to build the right UX for the type of relevance problem you are solving

By Nikhil Dandekar, Engineering Manager, Search at Foursquare

A few weeks ago, Chris Dixon laid out the idea maze for AI startups. Relevance products like search and recommendation engines are, in a broad sense, Artificial Intelligence products too. This means that it is very hard to be 100% accurate. Getting to 80% is relatively easy, but beyond 80–90% accuracy, we enter a zone of diminishing returns. You can spend years of efforts and will only be able to eke out very few points of accuracy gains.

Let’s consider just the first decision point of cdixon’s idea maze:

Given a search or recommendations engine having 80–90% accuracy, there are 1 of 2 things you can do:

1. Create a fault-tolerant UX
2. Narrow the domain and improve accuracy for the narrow domain

Now let’s look across the spectrum, from super-specific, navigational searches to broad recommendations, and figure out what the right decision to make is for each of these cases.

1. Navigational Searches

Navigational searches, are searches that have a very specific intent, and usually just one right answer. E.g. “population of san francisco”. For the narrow domain of navigational searches, you should try your best to get as close as possible to 100% accuracy. If you can achieve a high enough accuracy, you don’t need to build a fault-tolerant UX. You can just show the one right result in your UX.

Solution: Improve the accuracy for the narrow domain

Navigational searches don’t need a fault tolerant UX

2. Broad searches

Broad searches (e.g. “lunch”, “dinner”, “books” etc.) can have more than 1 right answer. If I’m in the mood for lunch, and type [lunch] in Foursquare, I might get a result list like this:

After looking at a few options in the results list, I might realize that I’m in the mood for Vietnamese food and result #3 is where I want to go right now. But for a search engine, knowing that result #3 is the most relevant to you given your current mood is very hard to figure out. This is where a fault-tolerant UX becomes important.

The classic fault-tolerant UX that search engines use, is to show ten or more results instead of just one. Search engines also show rich information for every result to aid your decision making while picking the right answer, such as:

1. Showing you “snippets”, i.e. text from the result page which matches the search terms.
2. Showing you other information that can help you make your decision. E.g. Foursquare will show you the distance to the place, the rating of the place, the price range for the place etc.
3. In the world of mobile apps you can easily add further interactions to the UX to make it even more fault tolerant. E.g. swiping away results that you don’t care about or easily shortlisting or saving results etc.

Solution: Create a fault tolerant UX specific to broad searches

3. Recommendations pages

Pure, search-less recommendations (e.g. local recommendations on the Foursquare home screen or movie recommendations on Netflix) are harder to get right than broad searches. You can only guess a very broad intent, such as visiting places or watching movies, and need to figure out the right answers for the user.

You again need to build a fault-tolerant UX, more fault-tolerant than what broad searches need. Given that you have the same amount of screen space as broad searches (a mobile or desktop screen), most good recommendation engines invest more in explaining (i.e. selling) the results to the users. Here are some ways you can do that:

1. Smart justifications that inform why the user might want these results. E.g. Similar to the movie X that you liked. Because you like witty, foreign movies (Netflix)
2. Group together results in specific, personalized sections. E.g. Recommendations for You in Automotive, Inspired by Your Browsing History (Amazon.com)

Solution: Create a fault tolerant UX specific to broad recommendations.

4. Push notifications:

Result push notifications are the at the broadest end of the search/recommendations spectrum. These are pure intent-less recommendations. The user never expresses any intent. Instead, based on the current context, the app decides that the user might have an intent at this point of time and pushes content to the user.

It’s very easy to pick and send a very poor push notification. So ideally, you want a really fault-tolerant UX here, with very clearly explained results.

But, unfortunately, that’s not a luxury you can afford. Given the very limited amount of screen space that push notifications get, it’s virtually impossible to build fault-tolerance.

So we have to go back to the other arm of the decision flow, and invest in narrowing the domain and improving accuracy within the narrow domain. For push notifications, it’s really important to be very relevant. Knowing the user and the context really well and sending notifications only when we are very confident that the user will find them relevant is the way to go here.

Solution: Improve accuracy by narrowing the domain to well-understood contexts.

One of the key takeaways here is that for relevance products, relevance (data) and UX are strongly interdependent. Building a great search, recommendations or notification product requires a close interaction between the relevance and UX disciplines. It’s very easy to fall into the design-first or data-first methodology for solving these problems, but the right solution often lies in close interaction between those two.