Representation in UX Research Participant Recruiting
Intro
DonorsChoose hit a milestone in our racial equity work in 2021, by launching our Equity Focus. Our Equity Focus shines a spotlight on schools with at least 50% of students who are Black, Latino, Native American, Pacific Islander, or multiracial and at least 50% of students who qualify for free or reduced price lunch. The distinction addresses the systemic racial and socioeconomic inequity that teachers and students face, while creating opportunities for donors and partners to support them.
For the UX Research team, our Equity Focus takes shape as the intentionality we put behind selecting the folks that participate in our research studies. Primarily, our goal in being intentional with our choice of participants is to ensure sufficient racial representation in user research. The participants that we select, as a research team, can have a significant impact on the decisions that we make about our website. We look for as wide a spread of participants as possible to get a diversity of perspectives because we aim for the website to be usable and useful for all of our users. The product team wants to ensure that we’re meeting our users’ needs equitably. Ensuring sufficient representation in user research studies is one of the research team’s most direct lines to the organization’s racial equity work and goes hand-in-hand with the org’s Equity Focus.
What are we trying to achieve?
Alongside racial representation, we aim for representation in age, gender, community type and geography. The demographic categories that we focus on may vary with the goals of each study. For example, usability testing for our new map feature necessitated that we pay closer attention to folks’ community type (i.e. rural, urban, suburban) to get a breadth of opinions on how the map appeared to them, which depended on their location.
Recruiting the widest range of participants possible isn’t always a straightforward path. The folks who respond to our surveys may lack the diversity that we’re ultimately aiming for — survey responses don’t guarantee a wide spread of participants. It goes without saying that trying to recruit a diverse group of participants from a homogenous group of respondents is a challenge. One practice that I find helpful to keep in mind during recruitment is to avoid the trap of only looking for a “perfect” candidate. Rather, I find that recruiting participants who may not check every criteria box but who can offer a unique perspective has been valuable. I go through a mental negotiation process while filtering through responses. For example, during our research studies for the map feature, a participant may not have answered a criteria-determining question in the most optimal way but they are from a geographic area that we have no other representation from. From there, I would ask myself and our UX Researcher, Josh, how the value of the criteria-determining question and that of the geographic question compare. Recruiting for representation can be a challenge because of the balance I aim to strike with participants: do they fit a good chunk of the criteria? Do they offer a perspective that we wouldn’t otherwise hear? Then more logistic questions come in: are they available when we are? If not, what is the extended availability that we can offer? And so on and so forth.
Addressing our racial equity goals
With all this in mind, our team was looking for a tangible way to address our racial equity goals. For now, we don’t have specific demographic goals. Rather, we decided to reflect on the makeup of our research participant pools each quarter. To keep an eye on this information, we’ve created a demographics tracker to do a statistical comparison of demographics by quarter to review the number of participants we met with from each demographic category.
The research team plans to meet to review the data and look for trends on a quarterly basis. After describing the trends that stood out to us (if any), we reflect on the numbers and how we got there. As a result of those meetings, we’ll reflect on whether we feel satisfied with the representation we had in our studies. This will look like asking ourselves whether we could have done more to have a wider range of folks (and if so, brainstorming recruiting strategies). As needed, we aim to establish strategies for increasing participation from underrepresented groups. All of that being said, the overarching goal of this work is to ensure sufficient representation in user research and to make sure we don’t leave out segments of our user base.
