Why Design Researchers Should Compensate Participants
Last year, Servis Finansye Fonkoze, Haiti’s largest microfinance institution, commissioned GRID Impact to assess its product and service offerings. As lead researcher on the project, it fell to me to determine how we’d recruit participants for the study — including protocols for compensating them.
If you’ve read my past writing or heard me speak, you know that I’m quite vocal about the need for research teams to compensate the people who participate in their studies, especially in vulnerable communities.
For one thing, it’s a matter of fairness. As a researcher, I am being paid, as is the driver who takes me to the research location and the interpreter who helps me translate. Why shouldn’t participants get paid too? Like all of us on the team, they are sacrificing their time in order to contribute to the research effort, and they often incur expenses too, like transportation to the research site or airtime to coordinate with the team. There’s also a significant opportunity cost, like time not spent caring for children or minding a store. In Haiti, some of the communities we spoke with were subsistence farmers, growing just enough pigeon peas, maize, and millet to survive. Time away from the farm can literally mean less food on the table. In more practical terms, compensation helps recruit participants, and serves as a very justified token of appreciation. We could not complete the project without them.
In fact, the relationship between research teams and participants could accurately be described as extractive, as Sasha Costanza-Chock argues in their latest book, Design Justice: Community-Led Practices to Build the Worlds We Need. No matter how inclusive, participatory, or democratic the process is, Costanza-Chock notes that the institutions doing the research and design always benefit more than the communities do. In a sense, compensating participants is one way to make the interaction a little less extractive.
As researchers and designers in the social sector, it’s true that we’re pursuing some kind of solution that ultimately benefits the communities we’re working with. But we can’t possibly promise individuals involved in the research that they specifically are going to benefit from its outcome. And it’s not realistic to depend on altruism as their sole motivation for research participation. We can’t even really guarantee that the project will lead to an improvement. As GRID Impact CEO Alexandra Fiorillo likes to say, “Just because it’s designed doesn’t mean it’s better [than what previously existed].” Going through the design process, no matter how meticulously or earnestly, does not guarantee that the resulting products, services, or systems actually bring about positive change, unless there is evidence and rigorous evaluation suggesting otherwise.
Despite these arguments, I still come across practitioners who are hesitant or opposed to participant compensation. This has led to many discussions with colleagues and peers over the years, and most of their responses boil down to five counterarguments. I’d like to address each of them here.
“Participants are happy just to have someone listen to them…”
They’re probably not as happy as you might think. In fact, many participants are exhausted. “Research fatigue” is a very real and well-documented phenomenon, especially in contexts with multiple agencies and NGOs. People in vulnerable communities may often feel that “repeated engagements do not lead to any experience of change” (as described in a research article by Tom Clark). Many participants we talked to in Haiti were quick to point out how many times they had been consulted, and yet they’d seen little as a result.
Researchers often like to think that the act of listening itself is somehow therapeutic, some touting that “participating was like therapy, in a way.” And while participants may appreciate being listened to, it’s dangerous and irresponsible to think of a research interaction as therapy. Mental health and counseling are professions requiring extensive training — training that few designers and researchers have received.
“Participants won’t give honest answers…”
Will a research participant give you different answers if you compensate or incentivize them? There are two questions I want to consider in order to evaluate this objection.
First, who exactly is changing their behavior? Do participants give more “satisfactory” answers because of the compensation, or are researchers treating them differently because they know they’re being compensated? A paper by Eleanor Singer and Richard A.Kulka, focusing specifically on how surveys are implemented, poses an interesting premise: “It is possible, for example, that interviewers expect respondents who have received an incentive to be more cooperative, and that they behave in such a way as to fulfill their expectations.” This suggests that at least some of the impact of compensating participants could be on the researchers themselves, rather than participants.
Second, will participants change their behavior simply because they’re being interviewed — regardless of compensation? The answer to this one is yes, of course. Most of us are familiar with the Hawthorne effect, in which individuals modify their behavior because they know they’re being studied. But the burden of minimizing this cognitive bias should certainly not be placed on the community by not compensating participants. Minimizing it is something good research protocols address.
In Haiti, for example, the GRID team drew on our behavioral science background to craft research questions and methods focused primarily on asking behavioral questions. This was done specifically to get at the intention-action gap of participants, and understand what factors were preventing them from adopting new products or processes. We also acknowledge the advantages and limitations of each research method. For example, independent activities like observations minimize the Hawthorne effect, while guided activities like interviews are more efficient at gathering targeted data. By combining multiple methods, we were able to triangulate our data to come up with a more rigorous and nuanced set of findings.
“It will make participant recruiting more challenging…”
If participants are self-selected, then it’s likely that offering incentives “will attract a subset of participants who are drawn to those rewards, making the resulting participants more similar in some dimensions,” as Gary Hsieh and Rafal Kocielnik noted in their paper. This can be a problem, or it may be a good thing, if the participants you attract are your target sample.
When participants are not self-selected, as was the case in Haiti, it’s up to the research team to handle the shift in dynamics that recruiting may introduce in the community. For instance, we took pains to be transparent about our sampling and recruiting criteria, in order to avoid people feeling unfairly left out when their neighbors were selected but not them. Explaining to the community that “we’ve heard many stories from X but we’re still missing the perspective of Y” not only helped them understand the rationale behind recruitment, but some even offered to help us find participants to fill that gap.
“Seeking informed consent will be harder…”
It’s definitely important to ensure that compensation doesn’t undermine the agency of participants, or violate the principle of voluntary participation. The Belmont Report, a seminal text in the field of ethics in research with human subjects, specifies that “an agreement to participate in research constitutes a valid consent, only if voluntarily given.” As such, informed consent should be free of undue influence, which “occurs through an offer of an excessive, unwarranted, inappropriate or improper reward or other overture, in order to obtain compliance.”
The U.S. Department of Health & Human Services offers additional guidance on the matter.
“[While] remuneration for participation in research should be just and fair, […] wherever the remuneration is set, it will influence the decisions of some more than others. In particular, it will be more important to those for whom it will make a significant financial difference.”
This tasks the research team with finding the right level of compensation: not so high that it interferes with participants’ ability to give voluntary informed consent, but not so low that it fails to fairly compensate them for time, expenses, and inconvenience. That said, in an article examining the legal and ethical dimensions of offering payment to research participants, Emily A Largent and Holly Fernandez Lynch raise an important question for us to consider.
“The various laws, regulations, and ethical guidelines that govern the conduct of human subjects research offer relatively little in the way of specific guidance about what factors or features characterize ethically acceptable offers of payment. […] It is, therefore, unsurprising that the space inhabited by IRB members and investigators is characterized by confusion and conservatism. […] That may lead to overprotection, and possibly distraction from things they should actually be worried about — particularly the possibility that offers of payment are too low.”
As such, it may be worth drawing “attention to the rarely asked, even radical, question: are research participants paid enough?”
“We couldn’t possibly give out money…”
Organizations in the social sector often have strict procurement rules, to prevent corruption and hold stakeholders accountable. Sometimes this means that clients or partners we work with are unable to set aside a budget for participant compensation. In these cases, it’s up to the consultant or agency doing the research to build participant compensation into their rate, as we did when working with Fonkoze.
With the budget in place, I believe — perhaps naively — that compensation should be in cash, or something close to it, like airtime or a gift card to an accessible retailer. This is so participants can spend their compensation as they see fit, rather than researchers paternalistically assuming what they need. However, handing out cash may not be possible or preferable for a variety of reasons.
Some are logistical, like access to payment infrastructure, particularly when sending money remotely. Others relate to the history of research with human subjects in a specific context, for example projects in the reproductive health space where there’s a disturbing history of paying for sterilization. In other instances, like in this case study by EPAM Continuum, working with the National Microfinance Bank in Jordan, Zach Hyman wrote that the team’s fixer/translator advised against paying participants in cash, as it “would be considered very rude.” Instead, the team assembled a box of incentives with items — almonds, dried apricots, chocolate bars, dates, and olive oil — that “were non-perishable, did not require refrigeration, and conferred status through an appearance of elevated taste amongst the recipients.”
In the case of our Haiti project, the fact that the client was a financial services provider made cash compensation to individuals who are its clients ill-advised, so we also put together a kit of incentives. Here, it was common household goods, including salt, soap, and detergent, that people would actually use. We also offered food and drinks during research sessions, and gave each participant a copy of their favorite photograph from the session.
Five Things to Consider When Choosing Incentives
When thinking about compensating design research participants, I found this list of criteria outlined by Alba Villamil a good starting point: value, fairness, liquidability (i.e., ability to be turned into cash or redeemed), sustainability, and appropriateness.
Are there other criteria that you consider when thinking about participant compensation? What are some ways you have found to work around procurement constraints? Are there any rebuttals that I might have missed? Feel free to comment below, or share with @SFath and @GRID_Impact on Twitter!
