Hello, and Thanks for All the Fish
Tips for effective research recruiting
It would be weird if a business owner said “I’m going to acquire 20 customers, and that’s it. Done getting new customers.” (OK, sure if each of those 20 customers gave you $100 million, maybe you’d rack up the cash and retire.)
Here in the real world, companies need to acquire customers continuously, more of them, different types, and bigger spenders — and also retain them. This means organizations need to continuously learn about their customers and prospective customers (and users, donors, supporters, etc.)—their beliefs, behaviors, preferences, and social relationships. Otherwise a competitor who understands the same people better is going to come along and snatch them away. And because we live in a world that changes over time in unpredictable ways, companies need to keep studying people even if they have been selling the same product to the “same” people for 100 years.
In order to understand the people you want to serve, you have to study them. To study them you have to get a representative sample. And this means recruiting.
If you don’t recruit a truly representative sample that means you’ll be studying the wrong people. Studying the wrong people is a colossal waste of time and money that leads to useless conclusions and terrible decisions. And yet, many organizations study the wrong people because crappy recruiting seems easier than good recruiting.
Because what is the one thing we know about people that doesn’t change? We’re all lazy, forgetful, creatures of habit.
So, here are a few things to keep in mind to avoid setting fire to a big pile of money°.
First, know what you need to know
Before you study anything or anyone, you need to know what you need to know. That will tell you who you need to talk to. Obvious, right? But companies ignore this critical first step all the time and just send staff out to accost randos in the Starbucks line. Or they turn to services that offer a cheap bucket o’ vetted users on demand.
Once you know what you need to know, then you can identify the best type of people to study to help fill in those gaps. There is no such thing as a “generic person”. You can be more or less specific, depending on your line of business and your goals, but different people will have very different responses depending on how much they’re predisposed to care about what you do. (Selling sandwiches or recruiting rocket scientists? Different!) And while you shouldn’t make the mistake of going too narrow too quickly—be intentional, not simply lazy.
You’re gonna want the wild catch
Availability bias is the fancy term for assuming that what is ready to hand is more representative than is actually the case. A very common example of this is assuming that the customers who contact support are more representative than they actually are. Oversampling the wrong slice incurs a lot of unnecessary risk.
So let’s imagine your business is fishing. And you want to understand the ways of fish so that you can catch enough of the right kind and do it sustainably. Someone might come to you and say, hey I run an aquarium. I have all these fish here in a tank for you to study. That’s sure convenient, but it won’t tell you much about the behavior of fish out in the wild. And sometimes the fish in those tanks are semi-professional parfishipants who turn up in a lot of other people’s ponds. Professional recruiters and recruiting tools promise a steady supply, but are they the right ones?
Red fish, blue fish, male twenty two fish
Do not recruit based on demographics. Do not recruit based on demographics. Yes, demographics are the easiest criteria to use. (Do not recruit based on demographics.)
Behaviors, motivations, and contexts are often correlated with demographics, but demographics do not determine behavior. And until someone manages to install a toll gate directly on a synaptic pathway, behavior is what businesses make money from. College students in America are likely to be between the ages of 18–24, so if you are designing a service for college students representative users are likely to fall within those bounds, but people in their twenties are not fungible. Most people in their twenties would be a bad match.
As you learn more and more about your various target customers or user groups, you can accumulate up recruiting criteria into a set of profiles. It is an iterative process. Learning is the goal after all. So do not get grumpy that learning is part of the process.
This is not to say that demographics have no place in recruiting. You use them to decrease sampling bias and get a good representative spread across age, gender, geography, etc., depending on (see above) what you need. So filter using demographics after recruiting using more meaningful criteria.
Use the right net
You have to use some sort of screener when you recruit, even if it is quite brief. A screener is a survey that helps identify good participants and filter out anyone who would be a waste of time. There are some tools designed specifically for recruiting, but you can use any survey tool, or even just Google Forms in a pinch. Creating good screeners is an art. You want to entice a lot of good potential matches to fill out the screener and be able to flag anyone simply seeking a payday.
The screener survey questions need to be short, specific, and possible to answer truthfully with very little thought. Good survey rules apply. No asking respondents to predict future behavior, speculate, or remember anything too far in the past. The ideal questions all depend on what attributes and past behaviors best correlate with the profile you seek.
The more you can verify easily with a quick web search, the less you need to ask. And do not let other people in your organization treat your screener like their research survey and tack on additional questions that don’t belong.
Go to the right spot
Once you know what you want to learn, and what population you want to study, you need to go where the right fish are schooling and start sampling. Hint: The place to be is probably not “jobs/etcetera” Craigslist. That’s one of the best places to find habitual study participants trolling for dollars, rather than real representative people you need to learn about. Nothing says “Send us warm(?) bodies willing to lie” like PAID FOCUS GROUP. (And I sure don’t fault people signing up to participate for the money. There are times in my life I would have been extremely interested in talking about pet food for $175.)
In every likely spot, drop a link to your screener with a brief description of what you’re looking for and the incentive. Something like this:
If you need to know about the people who come to your website, recruiting from your website can be swell, especially if you want to intercept people in the middle of a specific activity. Social media can be a great place to catch all sorts as long as you don’t need people who don’t use social media. (This is where the pre-research comes in.)
It is perfectly fine to go use your personal and professional networks, given a good screener. This is often the quickest path to high quality results because everyone along the chain is more motivated to distribute the request to help out. Advertising online won’t get you CFOs as quickly as finding friends of friends who went to business school. And using networks is more likely to get you fresh participants than skimming a recruiter’s existing database.
If you drop your net in one set of pools aren’t getting any bites, reflect on why that may be, and adjust your tactics.
Use bait, mate
Incentives are a tricky topic. You do have to offer an incentive, that is some form of payment in exchange for the participant’s time. However, it’s essential that participants have a genuine preexisting attachment to the topic or task you are interested in, rather than those who are primarily seeking out opportunities to participate in research for additional income.
In conjunction with thoughtful screening, incentives help to remove bias and increase the quality of your response pool. Without an incentive you will oversample people with a lot of free time, an ax to grind, or researchers studying other people’s processes. Some organizations have trouble offering incentives for participation because of procurement restrictions. If you want the right people to make time for you, you should find a way around it. Having an outside contractor run the recruiting and expense back the incentives works in many cases. Compensating people for their time and participation is the right thing to do.
We’ve used Amazon gift certificates with great success over the years. They are easy to manage for both remote research and in person studies. If you can do cash, everyone loves cash. Verify that whatever you are offering is of value to the participants in their local area. The nice people at Ethnio offer a handy incentive calculator.
The right amount depends on how much time and hassle is involved in the study and how specialized an audience you need. The quality and quantity of responses you get to your recruiting message are good feedback on your incentive. A dollar a minute is a fine benchmark for remote research. Maybe $100/hour for anything in person. If you want to talk to Hollywood defense attorneys, you’ll probably need to offer a little bit more. Be careful of offering too much. That can increase suspicion among people who are otherwise good candidates for inclusion.
And, by the way, as long as the participant holds up their end of the bargain—showing up and participating—they should receive the incentive. If it turns out they aren’t a representative user and you cut the session short, they get the incentive. If they are a terse communicator and give unhelpful one-word answers, they get they incentive. Participants do not pay for your mistakes in screening and facilitation.
Company schwag is not a good incentive. Come on.
Always be fishing
Recruiting, screening, and scheduling is the most time consuming part of ethnographic research—or anything qualitative that involves people. And it deserves to be done with care. The better your team gets at it, the less disruptive it will be. Plus, understanding what makes a good representative research participant is of a piece with understanding who makes a good customer, a likely user, or someone else whose needs and concerns you need to take into account. Let the continuous learning wash over you.
If you are continually recruiting, screening, and saving responses, you’ll have your own pool to reach into as needed. You can build your organization’s capacity for understanding people. You will learn how to spot and screen out the professional industry people who are researching your research, repeat participants who do it as a hobby, as well as those who are just doing it for the cash. And your research won’t stink.
My favorite recruiting tool is Ethnio. I like it so much, I’m an advisor, which means I get the pleasure of chatting with Nate Bolt from time to time. I receive no money for recommending Ethnio. I just think the people who make it are good people who want to help their customers do the right thing.
You can listen to a great conversation we had about the research process on the Voice of Design:
However, if your company wants to give me money to help you understand your business better and make evidence-based decisions, that is what I do. So, please, get in touch.
Happy fishing!
° The KLF thought it was cool when they did it, but of course came to regret it.
