Recruiting the best possible participants for your research
Tips on writing a great screener
The essence of great research is getting the right data from the right people at the right time. A screener is a questionnaire that helps researchers identify the right people — those who have the attributes and qualities needed for a specific study. Your chances of recruiting the right participants drastically increases by crafting a well-thought out screener. Without one you risk recruiting the wrong participants, getting the wrong data, and making the wrong decisions.
Every screener needs to:
- Identify the specific characteristics and behaviors of your target audience.
- Ask enough questions to gather the data you need before applicants tire of answering questions.
But the best screeners also:
- Provide insight to a applicant’s openness and descriptiveness
Define Behaviors and Attitudes
Years ago, I was having an incredibly difficult time recruiting participants for a study. After three weeks, I was still short, so I began to use referrals from previous participants. While I had a screener, it focused mainly on demographic information instead of behavioral characteristics.
I was able to recruit the remaining people, but during one of those sessions, it became very clear that the person was completely unfamiliar with the topic and was making up answers. Ultimately, I had to toss that data and spend even more time finding a replacement, resulting in a loss of time and money.
Clients can sometime start with vague or very large target audiences. For example, I’ve heard “everyone is our audience” or “let’s start with Millennials” many, many times. Screeners for these types of targets often rely heavily on demographic questions (gender, age, income, etc.) to understand the applicants.
And while it may seem intuitive, this is a mistake. Focusing on demographics means that not all your participants will share the characteristics of an ideal user. To get the best results, target people with the specific attitudes and behaviors of someone using your product.
Write the Questions
When building your questions, keep standard survey tips in mind and focus heavily on the following three goals:
Target the identified attitudes and behaviors
Let’s say we want to create a screener for upcoming research on Tab, an app that helps split a restaurant bill with others. One problem it aims to address is the difficulty of calculating each person’s share of a bill based on what they ordered. We want people who experience this problem so we would target those who:
- Eat at restaurants with other people
- Are in situations where the bill is not split for the table
- Wants to only pay for what they ordered
If we focus only on demographics, like “Millennials”, then we would get participants in that age range, but not necessarily people who experience this problem. So we would want to include questions to target behaviors, such as:
- How often do you eat at restaurants with other people?
- How often have you had to split the check for a group?
Gather enough completed responses
Like surveys, screeners are most effective when they are kept short, allowing for more people to stay focused and complete all questions. For example, a survey with three questions will always have a higher completion rate than one with 100 questions. While I haven’t found a magic number yet, I typically start at a maximum of 20 and pare down from there.
- Only ask pertinent questions — a good way to cull questions is to remove those that you could incorporate into the session.
- Use an inverted pyramid to structure your survey — most important questions first, demographic questions last.
- Use open-ended questions sparingly as repeatedly having to write long responses will drastic decrease the number of responses
Understand the openness/responsiveness of each participant
Someone’s willingness to talk, share, and describe directly impacts the depth and breadth of data you receive. Some applicants might have the right behaviors, but may not be as willing to provide the full, unfettered answers you want and need. Ask questions to understand how deeply or thoroughly a participant might respond during the actual study. Using Tab as an example again, we might ask questions like:
- Describe the last time you had to split a check. What did you do? What steps did you take?
- Pick three words to describe your experience.
If your screener is mobile friendly, remind participants to use the speech-to-text feature for longer open-end questions. And, if possible, have participants respond to one question using video or audio. This might decrease the total number or completions, but they provide unparalleled insight into the level of focus and detail your respondents will bring to your study.
Edit, Test, and Revise
Finally, be sure to edit and revise your screener once or twice. Test your screener yourself by going through it and submitting real answers. Then, send it to colleagues and ask them to do the same. This process will help you target the data you need, understand what it feels like to answer the screener, and cut anything extraneous.
Find more articles on research methods in User Research Resources.
Join us! Be a writer for User Research and write about the methods and tools you use, artifacts you create, and issues you face in UX.
