Tricks of the Trade: Conducting Research with Financial Service Providers
Senior Program Associate Dan Cassara speaks with Alain Shema (PhD Candidate, Syracuse University School of Information Studies) about his work in East Africa, supported by a grant from CEGA’s Digital Credit Observatory (DCO). Alain evaluates the impact of changing credit limits on small, digital loans for low-income borrowers, in partnership with an anonymous Financial Service Provider (FSP).
Collaborations between researchers and Financial Service Providers (FSP) are at the core of CEGA’s Digital Credit Observatory, which asks questions about how to design financial services to better serve and empower low-income populations. While co-creating research with a for-profit company can involve navigating sensitive things like data sharing, privacy, incentives, and market volatility, both the research teams and the FSPs involved in these partnerships have a lot to gain.
CEGA Faculty Co-Director Joshua Blumenstock puts it nicely: “Since the vast majority of real-world financial products are being offered by for-profit companies, it’s critical to engage those companies in the research process. This sort of collaboration can, on the one hand, provide the researcher with a realistic context in which to run experiments at scale. Perhaps more importantly, close collaboration can help ensure that the findings are relevant to the partner company and other financial service providers — and thus increase the likelihood that the results can improve future product design and business decisions.”
In the following interview, DCO grantee Alain Shema shares his experience partnering with an anonymous FSP in East Africa, and provides tangible insights for other researchers looking to do the same.
How did your partnership with the (anonymous) FSP begin?
With a little persistence and a lot of luck. From the very start of my PhD, I was interested in researching how mobile phone data could be used for the financial betterment of the poor. After completion of my required coursework, I returned to Rwanda and luckily learned about the FSP and their work on airtime lending from former colleagues.
Classmates from my master’s degree program were employed at the FSP and shared challenges they were facing, such as increasing default rates. I realized the overlaps between their work and my research interest, and proposed to collaborate. Luckily, the CEO saw the potential opportunities of this collaboration and quickly agreed to the partnership.
What were they hoping to learn, if anything, from your experiment?
One of the FSP’s main goals was expanding loan volumes without increasing defaults. While they had been collecting and generating huge amounts of data, limited data analytics skills available in Rwanda at the time had prevented them from maximizing the value of the data. They were open-minded about how I could design experiments that would help them learn more from their data and profitably reach additional customers.
What do you think is the value of embedding a rigorous evaluation within the operations of an established FSP?
FSPs, especially those working with people who have limited means, are constantly working to identify factors to help them target creditworthy customers who may not have been served otherwise. The better they become at identifying good customers–regardless of the customer’s income and/or wealth–the more poor people they can serve. As researchers, we are trained in identifying causal relationships from what may seem like noisy data. Hopefully, the rigor researchers can provide through scientific inquiry can help FSPs help more people in less time. The biggest challenge was thinking of experiments that would answer interesting research questions, while at the same time having the potential to make a concrete impact on the FSP’s business and customers.
Researchers can also benefit through this type of collaboration: the infrastructure built by FSPs presents enormous opportunities to conduct experiments at large scale, beyond the affordances of lab experiments and small scale surveys. In this environment, researchers can also better understand the interactions between their experiments and exogenous variables, such as countries’ socio-economic policies.
What lessons would you share with other researchers about running experiments with FSPs?
First, one needs to understand experiments from the FSP’s point-of-view: what are they trying to accomplish, and what would they gain from my work? As a researcher, you are very reliant on their collaboration. I quickly learned that I had to present any proposed experiment in terms of benefits to the FSP, as opposed to from a purely academic/research angle. In my case, all the funds lent during the experiments came from the FSP, who risked reputational and business damage if anything went wrong. Thus, I had to get their full support before anything could be started.
Second, one needs buy-in from the decision makers (i.e. the CEO or a department head), as well as the people who will actually build the necessary infrastructure to conduct the experiments (i.e. software engineers or data analysts). These are usually busy people working in a demanding environment. To ease the extra burden of the experiments, I tried to have my experiments automated as much as possible. Sometimes I would also share tools and techniques that could make their other work more productive.
Have you shared your results with the FSP and if so, how? Did they share any takeaways with you?
I engaged the FSP throughout the study period and shared preliminary results as well as drafts of my papers. They have been very receptive of the findings and we have been discussing how to implement some of the ensuing recommendations in their business processes. For example, one of my experiments showed that people who frequently bought airtime, regardless of the amount, tended to be better payers of airtime loans. While the FSP had previously looked mostly at the amount of airtime purchased, they are now planning to incorporate recharge frequency into their risk models.
What topics are you interested in tackling (or seeing others address) in the future?
I’d like to get more input from the consumers via surveys and/or interviews. My experiments were useful in isolating effects and providing answers to what would happen if you manipulate a variable (e.g., credit limit). However, I feel like qualitative studies would help answer questions around the why. For example, I am very curious to find out from participants their perspectives on how these changes influenced their behavior and, more broadly, their views on how best they could be served.
