When to go guerrilla with recruiting
Originally published by my friends from UX New Zealand on blog.optimalworkshop.com.
Everyone has heard of guerrilla research, the magical solution to getting feedback “in the wild” quickly and cheaply. Many in the Lean and Agile circles have latched on to guerilla research as the only way to do UX research. While I’m a huge fan of pretty much anything that helps us pragmatically collect and integrate research, guerilla research has its time and place.
Wait, what exactly is “guerrilla research”?
Let’s start by defining exactly what we mean when we say guerrilla research. The dictionary.com definition of guerrilla is:
- a member of a small independent group taking part in irregular fighting, typically against larger regular forces
- referring to actions or activities performed in an impromptu way, often without authorization
I certainly hope you’re not irregularly fighting against larger regular forces or acting without authorization, though as someone who has worked in non-research friendly environments, I feel your pain. The second definition better fits what we mean by guerrilla research.
In fact, guerrilla research is just a way to say that you’ve taken a regular UX research method and altered it in some way to reduce time and cost. This typically means that you also reduce the rigor of the method. The most common method to “guerrilla-ize” is usability testing; instead of recruiting participants to come into a lab setting, you can stake out a spot at a coffee shop and recruit random patrons to participate in short sessions or you can conduct the study remotely. You may also reduce the total number of people you talk to so that you can shorten the length of time it takes to get results. Guerrilla research has been praised as a way to integrate research into shorter and shorter timelines and limited budgets.
Things to keep in mind about guerrilla testing
Right so, guerrilla testing sounds pretty awesome. Why doesn’t everyone just guerrilla-ize everything if you can get good feedback in a shorter time frame?
Well, remember how I said that in order to get those time and resources savings, you’ll have also have to reduce the rigor of your research? That can be OK, but can also lead to some risks, especially when it comes to the participants involved in your research.
Remember that research participants are supposed to be actual or representative users of your service or product. Of course, whatever you build should be easy enough for most people to use, but you can miss out on some quality feedback if someone isn’t really a target user. Perhaps they won’t really care about your product or won’t have the same underlying knowledge that your real users would have.
For instance, I recently helped conduct usability testing of some tax software and we were trying to determine the usability of finding specific forms. Although we’d recruited tax professionals and thought they’d have the knowledge needed, none of our participants could find the form. We realized part-way through the sessions that our participants didn’t typically use that version of the form in their daily work. We couldn’t confidently tell whether they really had trouble locating the item or if they got stuck because they were unfamiliar with the form and process associated with it. We’d recruited and done some screening, but we still ended up with inconclusive results.
Even if you do have a product that doesn’t require specific background knowledge, you still may run into biased results. Let’s say you set up shop at a cafe to test first impressions of a new filtering process for shoes on an ecommerce site. You may talk to five people, but those five people are all patrons of a specific shop, in a specific neighborhood, in a certain time period.
Think about it; the visitors to a fancy cafe in a business area at lunchtime are going to be very different than the visitors to a cheap spot off a highway in the middle of the night. This means that you have to carefully choose your guerrilla location. No matter where you choose, you’re more likely to over-represent a particular point of view when everyone comes from the same context.
One other note about approaching people in public to take part in your studies; be aware that people are likely going to be surprised and a little unsure of what to expect. You don’t have as much time to build rapport or give context, and you might be in the middle of a public place. All of this means that people may be more hesitant than usual to be frank and forthright, especially when discussing sensitive topics. If your topic of conversation is health, money, sex, or relationship-related, random intercepting may not be the recruiting tool for you.
There are also plenty of online recruitment panels that you can use; some online tools have panels built in and some panels are separate. Independent recruitment panels often allow some screening and allow you to get quick turnaround, but you may also encounter some issues. Only a certain subset of people gets and takes the opportunity to be a part of a panel, and there have been lots of shared instances of the professional user research participant, who will “frequently supplement their income by participating in user research… and say and do whatever it takes to get into a study.”
Using an online research tool’s integrated recruitment panels, like Optimal Workshop’s group of participants, allows you to get quality guarantees about your participants. If you can tell that they lied about their qualifications or really aren’t the right fit, you can flag that person and get a replacement participant.
So when do I really need to carefully recruit?
Guerrilla research really is awesome, but there are some times when investing time and money into carefully recruiting participants will be worth it.
Here’s a handy chart to summarize when it’s really worth the investment in recruiting and when to go guerrilla:
Want to hear more?
I spoke about recruiting the right research participants at UX New Zealand. Check out the recording of my talk, as well as all the other awesome speakers!
Amanda Stockwell is President of Stockwell Strategy, a UX research practice focused on lean research methods and integrating user knowledge with business goals to create holistic product strategies. She has focused most of the last decade focused on finding innovative ways to understand people and embed that knowledge into overall process. She’s lead teams that provide research, design, and UX strategy services and frequently writes and speaks about her experience. Check her out on Twitter @MandaLaceyS and view her #LinkedInLearning courses on Agile UX Research, UX Research Basics, Card Sorting, and Interviewing.
Originally published at blog.optimalworkshop.com.
