
UX Research: Recruiting Test Participants
Going to recruit participants for your next great discovery and don’t know where to start? No worries, have a look here.
UX - U = X
(where “X” now means “don’t do it”)
Hoa Loranger ends one of her articles with one of my favorite quotes, where it’s pointed out how it’s just not possible to have UX without user research. This article will try to focus on one of the most common pain points for running tests. And what’s the main ingredient for running a successful test?
You guessed right: Test participants!
Criteria
It is never enough to stress how important it is to have test participants who are a representative sample of our final user group. Since there are cases where multiple user groups are needed, ideally test separately with each group. In order for the participant's insights to be relevant to the tests, a number of criteria must guide your recruitment. The criteria can be very general, covering things like age, location, and so on, as it can be specific as your requirements need. When deciding on these criteria, careful tuning is a must, between not being too broad that your user may no longer be representative or demanding to the point where it is impossible to recruit participants. When in doubt, you can always use the method “Extremes and Mainstreams” proposed by IDEO, where participants from both opposites groups are tested, and ensure the insights you get, cover a wide range of possibilities.
How many participants?
The number of participants will be adjusted according to the needs of the test. As Jakob Nielsen points out, the ideal number for a Usability Test is 5 users.
As for other tests, the recommendations are as follows:
- Usability Testing: 5 users should cover up to 85% of usability issues.
- Quantitative studies (aiming at statistics, not insights): Test at least 20 users to get statistically significant numbers; tight confidence intervals require even more users.
- Card sorting: Test at least 15 users per user group.
- Eyetracking: Test 39 users if you want stable heatmaps.
It is important to note that a surplus of participants must be recruited and treated as a normal participant, as there may be an incident rate, also known as a no-show rate, where life comes into play, such as accidents, illness, misfortune, and for these varied reasons, if any of the original participants become unable to attend may be replaced.
Screening
Remember when we talked about setting the right profile? Well, a screener is going to help you a long way to get the right fit for your research scope.
Think about a screener as an initial survey, composed of a number of questions, that may be as simple or as complex as the range your research will cover and will allow you to find which candidates possess the characteristics you’re looking for. The criteria used for this questionnaire should reflect the different user groups, so specify the criteria to the point where the description states how many users have each attribute, to include the entire range in the overall number of participants.
As such, a screener is a great way to ensure the poll of participants attending the test will be an effective representation of the potential final users (hopefully some of them might already be).
Where to look for?
It “depends”. Of course, the easiest way would be to have access to a database with all the users and their specific characteristics. But this type of solution requires greater investment to establish a systematic recruitment process. In time, this process makes the research recruitment easier.
If there is no access to the research panel, you can always use a professional recruitment agency. But still, in case the budget is not sufficient to cover this service, there are other options. Social networks are a good base for starting to advertise these opportunities, either through direct networking, or using the company’s own networks. You can also go to the sales and customer service department, which as a rule will be the most in contact with end-users, they can have valuable leads in the recruitment process. If a particular user group is difficult to reach, similar users may be asked for referrals, or look into online interest groups and forums. Even if these resources are not available, or do not provide the answer, you can always recruit internally.
Keep in mind that some of the approaches may include the risk of recruiting participants who are close to the project, or that from personal relationships, maybe be biased test participants.
Incentives for participants
Usually, some type of compensation should be awarded to test participants to make up for the time spent on testing. This compensation does not necessarily have to be monetary, something of similar value can be attributed, whereas in some cases a cash reward may not even be possible. This will all be discussed.
When determining how and how much this reward will be offered, we have to consider different variables. First, and this may have more impact on small businesses or companies with a low research culture, we have to realize what our limitations are: What is my budget? What can we offer with this?
Other variables to consider, for example, are the time spent on each test by participants, the complexity of the tasks requested, the level of professional proficiency, and so on.
When it comes to financial rewards, one way to think about it is to understand what that user already receives in their professional environment and what would motivate them to bother to participate. Let us remember that the higher the level of a particular trait or skill, the greater the reward to be offered.
On the other hand, we have some non-cash options that are still highly valued by many users. Sometimes it is even easier to offer something that is already at hand, a small gift, from a product sample, to access something premium or even a discount on a purchase. If these options are not available, something unrelated to the business is positive too, a lunch, a book, movie tickets, the options are plenty.
This is especially true if test takers cannot be rewarded with cash. As is the case with some government employees or so. Even if some companies won’t pay for the time spent by internal employees, hey at least give them chocolate :)
Further considerations
Location
This can have a major influence on test participation, making the site easily accessible with parks or near public transportation. No-show rates may fluctuate depending on the conditions offered. Think about adding a map, or directions to the venue, in your invitation. When it comes to researching users who have mobility limitations, or if these observations have to be made on-site, travel to them may be necessary.
Invitation
When writing an invitation to participate in a test, you may want to keep in mind a few points. Include some basic information as to expect from the session, but try not to reveal too much information to avoid any biases. Clearly state what will be the incentive offered to perform the test. Ask the best way to get in touch, this may even prove to be useful information for this test, as for future tests. Here would be a good opportunity to request permission to add this information to your database. Whenever possible, include the screener in the invitation, so it may be possible to save time.
For further information check these:
- The Basics of Recruiting Users for Usability Testing
- Recruiting Test Participants for Usability Studies
- Recruiting Usability Test Participants
- UX Without User Research Is Not UX
I hope this post is helpful in your UX Research adventure!
‘Till next time!
