Researching with products destined for use in noisy contexts

The products we design at Deliveroo are destined for use in a multitude of noisy contexts, whether that’s busy restaurants taking orders during peak service, riders navigating to a customer’s address whilst wearing thick motorcycle gloves in the rain, or customers ordering for a crowd of hungry football fans on match day.

As researchers, we’re mindful that researching in a lab setting means losing some of the noise, constraints and frustrations of everyday life.

This becomes especially pertinent when we’re testing features that are designed to help users when things are going wrong, when people are likely to be feeling more frustrated and impatient than usual. Usability testing these features with calm participants in a quiet lab, for example, wouldn’t be a fair test of how usable these features really are. For features like these, context matters a lot.

One of the challenges we face as researchers is finding creative ways to combine the richness of environmental factors that come from researching in situ, with the benefits of the controlled and clear insights that come from researching in a lab setting.

Netflix and Deliveroo, a match made in heaven

One way to do this is to replicate contextual factors in a lab. We used this technique while looking to better understand how people choose where to order from. We knew anecdotally that people often order with a group of friends, and that having other people round can affect how they choose food.

Something that should be fun can quickly become a painful experience when you throw 6 rumbling tummies into the mix, one with a nut allergy, one recently converted vegan and another that’s already had pizza three times this week and is “so over it”!

After hearing lots about this in interviews with customers, we wanted to see for ourselves how these decisions play out and where there might be opportunity for improvement.

To replicate a more realistic ordering experience, we recruited one Deliveroo customer and asked them to bring along 5 of their friends to the session. We transformed a sterile usability lab into a cosy living room and left them alone to sit on the sofa, catch up, watch Netflix and order food together.

Once we got over the weirdness of watching people hang out, we were able to observe lots of latent needs that we would have missed if we had relied on their self-reports alone. In the debrief interview, for example, the group described choosing what to order as a pretty quick and easy process, when in actual fact we observed that it had taken them over half an hour to pick a restaurant. At one point, things had gotten so tense that to settle a decision, they even wrote their individual preferences on little bits of paper, screwed them up into balls and drew a winner out of a wine glass to pick a cuisine! This experience is something we would have totally missed if we’d just asked them to report on their behaviour.

When it’s the context itself that matters most

For other projects, it’s more important to observe behaviour in the user’s own unique context.

About a year ago, although we had done lots of testing with our customer app in a lab setting, we wanted a deeper understanding of what our customers go through while they’re waiting for an order. We soon realised that for a hungry customer, order tracking can be an emotional experience that would not be well replicated in a lab. The best place to get this kind of insight would be in the moment, sat by our customer’s side in their homes as they order.

One of the participants eagerly awaiting his order.

The findings were fascinating. They produced illuminating insights for the designers of the order tracking screen and gave the team a far more complete understanding and empathy than if we hadn’t ventured outside the lab. For this project, the contextual factors were the most valuable source of insight, watching customers use the delivery estimate to decide when to heat their plates in the oven or when to give their children a bath showed the team how important clear order tracking information really is.

As with any research method however, there are drawbacks to this approach. As great as it is to observe people in the moment, it can be very time consuming and isn’t always the most practical approach for the question we’re trying to answer.

Taking the streets to the lab

Take for example, recent testing on a new feature for our rider product, We knew from riders’ anecdotes (and our own experiences out there riding), that it can be incredibly frustrating when you’re looking for an unfamiliar address and the map pin isn’t in exactly the right place. However, this isn’t something that happens very often and so setting up a test for this in the field with users would have been tricky.

However, usability testing a feature that was designed to help in this scenario with participants sat in a lab was also proving difficult. A core part of this feature involved the frustration of your physical location being at odds with what you’re seeing on your device. No matter how many different ways this scenario was described, participants were really struggling to place themselves in this mindset.

To better test this, we took inspiration from the world of board games. While interacting with the prototype on a phone showing an order coming in, participants were also asked to trace a small Monopoly counter across a physical map of the city’s streets, to show where they would go and how they would get around.

This is what the ‘board game’ looked like. The participant is moving the counter across the map to describe how they would get to the restaurant from our location.

Interestingly, the presence of the physical map task helped unlock tacit knowledge the participants had acquired to help them navigate around town. Participants started to explain which junctions they knew to avoid, the quickest way around one-way systems, and which roads would almost certainly be gridlocked at this time of day. Although this was still an artificial setting, in contrast to the verbal context description, this task helped participants to better contextualise themselves in the scenario and brought them a lot closer to the kinds of frustrations a user encountering this issue in real life might experience.

Wrapping it up

Conducting research for fast-moving product teams means we have to be tactical when designing studies to better understand how people use our products. Balancing the time-expensive approach of full contextual research with getting realistic findings in a speedy manner requires careful thinking about the validity of the environment. The examples here are just some of the ways we’ve been striking that balance between context and scalable insight.

I’d be keen to hear from others on what they’ve done to strike that balance in the comments or via email. And if you’re interested in roles here, we’re hiring — get in touch!