Usability Testing gone wild

Usability tests have become a given…

Usability testing is crucial for building a product that is easy to use. You know that. Product Designers and UX Researchers everywhere know that.

Here at BlaBlaCar, we know that too.

But we also know success depends on moving rapidly from idea to implementation to iteration.

Going guerrilla

To get real feedback, from real people, real fast, we’ve adopted guerrilla testing. It gets us instant feedback on our ideas.

We bring our designs to public places such as coffee shops and ask random people if they would give us 5 minutes to share their thoughts. And more often than not, people say yes! They love the idea of looking at designs “in progress” and sharing their ideas… especially when you buy them a coffee.

Once someone agrees to take a look at the app, we pull up a chair to observe how they interact with the design and listen to what they understand.

• We hear first-hand if content we wrote is confusing
• We see directly if users struggle to complete the task

In less than an hour we have the reactions of 4 or 5 people and a much clearer idea of the direction we should take for next iterations.

And the next day, we do it all over again.

Does it actually work?

Guerrilla testing can’t tell you if your project or design will be a success or a failure, but it can help you make informed decisions about the direction you want to take. It can also help avoid small obstacles that you might have otherwise overlooked.

Here’s a simple example of a quick fix we made after guerrilla testing a flow asking drivers to share their license plate information.

Before and after: From “date” of your car to “year” of your car

We noticed that when we asked people about the “date” of their car, they hesitated about what format of number they should enter: mm/yy? yyyy? We changed the language to simply ask for the year and solved the problem.

The dangers of the wild

Not recruiting users means we can be fast and flexible about our testing schedule. But it also means we can’t be sure that the “testers” are the right target.

For a usability test in the office, we recruit very specific populations, such as people with cars who have used BlaBlaCar at least 10 times and who have a trip planned in the coming month.

In Starbucks or a train station, we get…people.

We can only ensure a bare minimum of criteria. For example, for the project above, we looked for car-owners.

We also can’t spend a full hour with a participant the way we can in our office for a “traditional” usability test, so the flow we’re testing needs to be quick! We need to focus on answering a single question.

Guerrilla best practices

After a couple of months of putting this method into place, we’ve developed some techniques to get the most out of it:

• Go to places where people have time to kill — train stations, coffee shops, parks, etc.
• Nail your opening line — people will be hesitant at first, convince them in the first 30 seconds
• Screen for a minimum criteria — make sure the topic is at least a little relevant for people you are talking to
• Go in pairs — one to lead the “interview” and one to take notes
• Don’t get discouraged — some days will be harder to “recruit” than others, try again tomorrow!

Get out in the world!

At BlaBlaCar, we believe that “The Member is the Boss” — meaning that the people we work for are outside the walls of our office. Guerrilla testing is a powerful and efficient method for us to get insights about what could really work “in the real world”, beyond the limits of our meeting rooms.

All for the cost of a coffee.