How to do guerrilla usability testing right
5 tips to help you get started
People talk a lot about usability testing and how important it is to get feedback from users. In theory, I totally agree, but in practice it can become tedious and overwhelming very fast (especially if you’re new to this field). Here is what I learned from my experience with guerrilla testing on an iMessage app that allows people, mainly teenagers, to send each other challenges.
1. Be well prepared
Guerrilla testing doesn’t mean it has to be messy. If you want efficient results you need to optimise your tasks and metrics. I recommend spending a good two hours to think about your goals and how to mesure what you want to test. Improvising is good, but if you don’t even have a basic outlined process, you will end up having chaotic results and spend way too much time per participant. It’s easier to find people willing to spend 5–10 minutes of their time with you than 30–40 minutes.
Don’t ask participants for more than 10 minutes, respect their time.
In my case, I prepared a scorecard, to make sure I stayed focused on my metrics and the success rate of the different tasks performed. It also helped me share the time between writing and observing my participants. And later on, I could easily transform my score card into measurable success.
2. Go to the right place at the right time
If you want to get the most relevant feedback, you need to find the most relevant participants. Don’t just wait in a Starbucks café with a sign saying “free coffee for a 10-minute test” if you’re not testing an app for coffee drinkers. You need to be moving and find your users where they are, don’t wait for them to come to you.
Don’t just wait in a Starbucks café, go where your real users are.
For my test, I first decided to go to a train station, in the afternoon, hoping to find a lot of people willing to test my app (as they were waiting and had nothing else to do). That was a bad idea :( I could only meet seniors or families going on a weekend trip and nobody from my target group — teens! I then walked around the streets for 10–15 minutes but it’s pretty hard to stop people in the middle of a sidewalk. I was lucky enough to end up where actual teens hang out when not in school — a skatepark — Bingo !
3. Keep it short
You want to test as many participants as you can. So keep it short and sweet, don’t try to test everything, stay focused on one feature, and one feature only! Remember, you can always run another session for a different feature. Don’t forget to get a minimum amount of data per user (age, device…) — this will later help you separate noise from relevant feedback.
Don’t try to test everything, focus on one feature only. Tweet this
The app I was testing needed two participants. One that sends the challenge and one that receives and responds to it. I couldn’t have one user test both sides. I then switched between testing with one person how to send challenges and with the next one how to respond to the challenge.
4. Team work is always better
Going solo is totally possible but everything will happen very fast and you don’t want to miss on any precious feedback. I suggest you do it as a team of two, one is in charge of the talking, the other one observes and takes notes. Don’t underestimate non-verbal communication, a facial expression or a hand gesture can sometimes tell you more than words. As much as possible, stay quiet while watching participants get to know your app and silently evaluate its usability.
Don’t underestimate non-verbal communication, it is as important as the words they’ll say.
For this guerrilla session I was introducing the tasks and paying close attention on their behaviours while my teammate was taking notes (observations, success rate…)
5. Stay open-minded & be aware of biases
When the session is done, you’ll probably end up with tons of feedback and not always what you’ve expected. Sort through all the different answers you’ve got and don’t neglect a comment because it was mentioned only once. On the contrary, this might be the most precious piece of feedback you’ve gotten today. It is not the popularity of a comment that makes it more relevant. Take into consideration both the people and the context. Make sure to keep the biases away by using open-ended questions like: How would you do that? What do you think you’ll see when you do this ? What do you expect to happen ? These types of questions allow people to tell you how they perceive your app without suggesting any answers to them.
Don’t use guerrilla testing to try to prove your own assumptions.
One of the best feedback I got, this time, was how a group versus a single person could change the satisfaction and increase the excitement of the game I was testing. When a group used the app, people were more creative and had a lot more fun, they were sharing their challenges and laughing at each other performing them.
And one last advice: Do it as often as you can ;)
Any more advice to add to this list? Feel free to write a comment below and share your own guerrilla testing experience! How often do you do it? How does it impact your design process?
You want to improve your app?
Join the new meetup : User Speed Testing #1
