Designing with Data to Improve Vimeo Search

I worked with the Search Team at Vimeo to create front-end designs for three new search features: Language Detection, Spell Correction, and Error States. In this case study, I will be talking about my process with designing new Error State pages.

My Role: UX Researcher and Designer

My process involved: a competitive UX audit of search systems, synthesizing secondary usability research, analyzing Vimeo’s collected query data, exploratory design mockups, setting up A/B tests, production designs, and documentation of research findings and recommendations.

Identifying Problems with the Current System

The task at hand was to redesign Vimeo’s existing Error State, which is currently composed of Vimeo’s null state Robot, along with some random popular videos below the Error State Message.

A successful Error State should have the following:

  1. Clearly communicate that no results have been found
  2. Offer suggestions to refine the query
  3. Provide a path forward, not a dead end

The current design is problematic because it is not immediately clear that no results have been found. Although it may seem a stretch, the videos that appear on this page could be mistaken for search results, especially to new users of Vimeo’s search, as they are much too close to the error state message. However, the current design does a good job at offering suggestions for query refinements (“try searching again using broader keywords”) and offering a path forward to popular, albeit irrelevant, Vimeo videos.

Exploring Solutions

An easy fix to misleading our users with random irrelevant videos, would be to add some spacing between the error state message and the videos below. This design makes the page look a little more empty, and does a good job at focusing the user’s attention on the error message before they see the suggested videos.

Alternatively, a solution is to completely remove the suggested videos so as to create no confusion whatsoever. However, this design suffers because it does not provide any paths forward for the users — it is a dead end.

Using Data to Find the Best Path Forward

I worked closely with Ryan, a PM on the search team, to collect and analyze data about our user’s search behavior. Ryan was able to gather a list of all queries conducted on a single day as well as get a breakdown of the type of user who uses Vimeo search (logged in vs. logged out; paid vs. unpaid). Using this information, we came up with the following assumption:

Most of Vimeo’s searches are coming from new users who are unfamiliar about the content Vimeo offers.

Over 98% of Vimeo’s searches come from logged out users and most of their queries concern mature content. If you filter the mature searches, the rest of the searches are broad terms that can fit into one of Vimeo’s many Categories (Animation, Comedy, Narrative, etc.).

The insight gained from our user data pushed us to explore the idea of driving our users to Vimeo Categories, which are effective exploratory gateways to high quality Vimeo content. Error states are a great opportunity to educate users who are likely confused or misinformed about the type of content on Vimeo.

By providing our users with a glimpse and easy access to Vimeo’s Categories, we are able to teach them about the type of content on Vimeo, as well as entice them to explore some of the videos in the categories.

A/B Testing

We are currently A/B Testing the effectiveness of Vimeo Categories in Search Error States.

Improving Search Beyond Error States

The insight from our research urged us to explore ways to drive our users to Vimeo Categories through other means beyond Error States. Our search currently returns results based on literal string matches. In other words, if a user searches for “funny” (the fourth most searched term on July 7, 2017), they will see results with the term “funny” in the title, such as a video called “Funny Bunny”. However, users would probably be much happier if they were presented with funny videos instead. This is a perfect example of when Vimeo Categories would be much more effective at introducing content.

By clearly suggesting a category to our users who search for terms that are related to one of Vimeo’s Category pages, we are able to balance navigation with search and provide our users with relevant content that are closer to their true desires.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.