5 steps to better debriefs with your user research observers
I properly love it when colleagues are interested in UX and come along to observe user research sessions. But I’m so busy looking after the participant that I sometimes forget that it can be a bit daunting for the observers too.
They aren’t familiar with the format, they don’t know what to look out for, and it’s a bit of a shock when we ask them to share their observations and insights at the end.
I’ve tried lots of different ways to solve this. There’s no perfect answer and a lot depends on who’s in the observation room, but I’ve found a few key pointers that lead to more engagement from observers and more valuable notes at the end.
1) Focus-it with Post-Its
When we let the observers just freely chat about what they saw, we see similar problems as in classical brainstorming.
- Loud voices take control, so quiet voices are silenced
- The group gets too hung up on one or two threads
- The group jumps into arguing about solutions to the problem they think they saw instead of articulating the actual problem.
Before long, we have a room of people arguing about why things are the way they are and why this or that suggested solution won’t work.
We don’t want this.
Instead, we give each participant a stack of post-its and a thick felt-tip pen. We tell them to note one observation per post-it. At the end, they’ll stick their post-its up and then we’ll discuss.
But what should they note down?
2) Tell them to watch for just one thing
Watching for everything that happens in a usability test is a big ask, especially for people who haven’t done much observing before. Corissa, one of my UX colleagues, recently suggested a focusing question that seems to really work.
We tell observers to look out for one thing and one thing only: EMOTION.
Why? Well, as Cindy Alvarez points out in the excellent Lean Customer Development:
Emotion = Priority
So we tell our observers, “look out for when the participant gets emotional. What was the emotion? What triggered it? What happened next?”
3) Tell them exactly what we want them to do
We send an email on the day of the testing that covers the pointers, then remind them of the critical ones just before we begin each session.
- “Sit and fully experience what the user is experiencing.”
- “Note down what actually happened and where (emotion, trigger)”
- “Note down the exact words the user said”
(yes, we care what they do much more than what they say, but sometimes we get valuable voice-of-customer insights from quotes.)
4) Tell them exactly what NOT to do
- “Don’t tell stories and interpret what you think is happening — just write down what actually happened”
- “Don’t come up with solutions right now — there’ll be plenty of time for that all the rest of the time you’re at work”
- “Don’t bring laptops and phones”
5) Debrief straight away with silent stickies before chat
We debrief right after the testing participant leaves. The half-life of observations in brains is short and insights start decaying the moment the test is over.
First, we ask the observers to stick up the post-its they’ve collected. This could be onto print-outs of the screens we tested, or just onto one of the few walls or windows that isn’t already festooned with Post-Its.
While we stick, there’s no talking! Only once everyone has contributed do we open the discussion. We try to follow the order of the session.
It’s always interesting to find out what different people noticed or missed – just how much different observers bring different perspectives.
After we’ve reviewed the post-its, it’s no problem if we veer off-topic into solutions and bunfights.
Let me know if you try any of these. And I’d love to hear if you’ve found any other pointers that help get better engagement and insights with usability testing.