Getting lost, no shows and asking participants to leave…

Everything we’ve learnt from User Research at Play — Part 1

There’s lots of different ways you can do it - focus groups, task based testing, interviews to name just a few. We tend to vary our approach from project to project using both quantitive and qualitative methods to inform our work. But. The main, and most important thing with user research is that you get your ideas/prototype/designs/products in front of people as often and as early as you can.

“Great design doesn’t live inside designers. It lives inside your users’ heads. You get inside your user’s heads by doing good user research: research that provides actionable and testable insights into users’ needs.”

David Travis, The 7 deadly sins of User Research

So heres my tips for running successful qualitative user research sessions, with a few minor disasters thrown in…

Give clear, specific directions to your office

If you’re conducting research in your office or a research lab provide crystal clear directions to your location. We’ve had problems with people finding our office before and have had to go and retrieve them. If you need to take a photo of the front of your office, or write a small essay with intricate details about how people can find you, do it. No detail is too small and it will mean you don’t lose valuable time. Heck, if you can provide a list of all the nearest car parks and how much they cost, do it. Because if a participant has spent the last half an hour trying to find a car park near your office, they’re not going to be best pleased with you.

Be flexible

Respect the working constraints of your socio-economic groups. Different working patterns, transport options and childcare availability can mean weekday research sessions just aren’t convenient. Work around the type of participant you’re looking to recruit and if that means an after 6pm research session then that’s what you need to do. Also, trying to recruit for the Friday before a bank holiday is pretty much impossible, so don’t bother. Check your calendar and avoid all the big holidays and sporting events.

Participants will be late sometimes

We’ve all been there, you arrive at your destination in plenty of time, go to park up and have to download an app to pay for your parking. But you’ve got no space left on your phone, so you have to delete a load of photos to make room, but then iCloud isn’t playing ball. When you actually get into the parking app, you have to create a password, but apparently you’ve already registered with that email address before… and now you’re late.

Participants will be late sometimes, and that’s ok. Do as much as you can in the time you have and if you’re lucky your participant may stay a little longer to make up the time, just make sure you politely ask permission to do this. If not, getting half way through your prototype or concept cards is still valuable. It’s not a disaster and your participant will already feel bad for being late, so make sure you’re extra friendly to make them comfortable. If you absolutely cannot conduct your session in the remaining time then speak to your recruiter and see what they can do to rectify the situation.

Occasionally, participants won’t turn up

Again, this happens. Sometimes people get called into work last minute and have to cancel. Sometime’s they don’t let you know they’re not coming and turn their phone off. It’s not ideal but just make sure when you’re recruiting you account for this. We’re lucky in that the recruitment agencies we work with try to back fill for us if this happens. We normally test with around 5 participants (Why you only need to test with 5 people by Nielson Norman Group) so we tend to recruit for 6 or 7. We’ve also found different socio-economic groups have a varying drop out rates, so do your research, talk to your recruiter and plan in contingency.

Recruiters will let you down sometimes

We worked on a project where we specifically needed to recruit females for a group research session. Myself and Ryan were all ready to go, 3 out of the 6 participants had arrived, and then 2 males turned up… We had 2 choices, run the session with them or ask them to leave. After a little (quiet) debate in the kitchen we decided to ask them to leave. However as myself and Ryan were running the session, doing this could be awkward and at worst inhibit the other participants. So Charlie our project director took care of it. He politely explained the situation to them, apologised profusely for the mistake and we were able to run our group session to great success. Mix ups happen, but make sure you consider how you handle them carefully. Facilitators need to play the good cop to make sure everyone is comfortable enough to participate in the session

Don’t be scared of what you might find

Sounds a little dramatic, but your job as a facilitator isn’t to convince participants of the product you’re working on, it’s to get to the truth. If that truth is that they wouldn’t use this product then great, you’ve discovered that now as apposed to when it’s been built.

Don’t use the word ‘test’

‘Test’ implies being right or wrong and with user research there aren’t any right or wrong answers, just honest opinions. So don’t invite your participants to a ‘User Testing Session’, it’s a ‘User Research session’. Also, make sure your recruiter communicates the nature of the session clearly to all participants. Turning up for a group session only to find it’s a one-to-one interview could leave people feeling a little put out.

You didn’t design the prototype, even if you did

If a user think’s you’ve produced the prototype you’re showing them, they’ll be worried about causing offence and you won’t get their honest opinion. So do your house keeping at the start of the session. Explain it’s not a test and there are no right or wrong answers. Honest opinions are greatly appreciated and you didn’t work on the prototype so you’re not going to be offended by anything they say.

NDA’s can take a while to read

If you’re running a 60 minute session and 10 minutes is taken up by a participant reading and signing your forms, that’s valuable time you’ve now lost. Do your prep and get these documents emailed over well before the session and save yourself one sixth of your valuable research time.

Wear a watch

Striking a balance between cultivating valuable discussion and getting through your whole prototype in 60 minutes can be difficult. If you’ve got a participant talking about one of your features that’s great. You absolutely don’t want to cut them off because that may deter them from opening up again. But equally, you’ve got lot’s of other features to get through and you have just spent 15 minutes of a 60 minute session discussing one. So keep track of time and if you need to, prioritise. If it’s looking like you may not get through everything — no problem. Know what you absolutely must cover and make sure you get that done. If necessary, thank them for their contribution on that idea and explain you’re conscious you need to get their opinion on a few more things.

Get someone else to take notes

If you’re busy writing down what someone is saying, you’re not looking at them and if you’re not looking at them you can’t gage how they’re feeling about your prototype. Get someone else to take notes for you so you can focus on having a valuable conversation with your participant. We’ve found post-it notes and sharpies work best for note taking. Marking notes with happy, indifferent and sad faces can speed up going back through findings later on.

That just about covers getting your participant to the session and some of the prep work you need to do beforehand. In the next part we’ll look at tips for during the session and how to handle almost every eventuality…

Fancy a chat about this post then email us hello@aplaything.com or check out our website www.aplaything.com