The Busara Blog
Published in

The Busara Blog

In a manner of speaking

Using IVR to measure accent bias

By Ruth Canagarajah, Samuel Oyegunle, Leonard Waweru and Elijah Baraza

Illustration by Micheal Bagorogoza

Following a battle with the Ephraimites, and in a bid to assert their dominance, the Gilleadites are said to have set up a blockade along river Jordan to capture fleeing Ephraimites trying to get back to their territory. The sentries asked each person who wanted to cross the river to say the word shibboleth. The Ephraimites, who had no sh sound in their language, pronounced the word with an s and were thereby unmasked as the enemy and slaughtered. The word shibboleth is now commonly used to refer to a form of linguistic password. A way of speaking that will identify one as “in” or “out,” and how they will be treated based on this.

Accents are like everyday shibboleths. Their consequences may not be life or death as it was for the Ephraimites, but our voice carries ahead of us, triggering unconscious bias in the world around us. The thing is, most research is done by voice. How can researchers trust their data if their presence is affecting it?

The problem with accents is that they’re very difficult things to typically control for — and they aren’t the only things that affect how people react to voice. Tenor, warmth and gender are all factors that come into play — especially when people are talking conversationally. We had to find a way to ensure that we could see how multiple people would respond to the same conversation.

Enter Interactive Voice Response (IVR) technology. IVR uses a computer- generated voice to ask survey questions by telephone. IVR allowed us to focus on one researcher reading the script carefully, meaning that all respondents would receive each question, response option and instruction in the exact same way.

Armed with our script read by the same voice in three different accents (American, Luo and Indian) we went to conduct a recruitment exercise with Kibera respondents. We were looking to investigate and measure implicit assumptions with accents and service provision. The script was designed to assess differences in financial expectations, willingness to partake, trust, and self-stated demographic information

Kibera is, arguably, a demographic group of respondents most targeted in Kenya for international NGOs/researchers and subject to the oversampling penalty (over-researched contexts that change the types of responses and incentives of populations partaking in research) to investigate and measure implicit assumptions with accents and service provision. Would we see the observer effect if we exposed respondents to the different accents? Does the voice delivering the survey test have an impact on the result?

It might. Our study shows that:

Respondents indicated very high financial expectations when exposed to the Indian accent as compared to the American and Luo accents. Respondents exposed to the Indian accent had the highest average earning expectation of KSh 1,971.44; those exposed to the American accent indicated KSh 1,536.82, while those exposed to the Luo accent indicated an average of KSh 1,522.28.

Kibera respondents also reported the highest cases of unemployment and low monthly income with the Indian accent compared to the American and Luo accents. The reported unemployment rate was 68.9% among the respondents exposed to the Indian accent, 64.4% among those exposed to the Luo accent and 52.8% among the respondents exposed to the American accent. Our evaluation accounted for 207 participants that were exposed to the American accent, 208 to the Indian accent and 199 participants exposed to the Luo accent.

We all speak differently. The phonological rules that underlie our speech patterns are like maps to our past. People listening to us bring their perceptions of what who we are means to the questions we are asking — influencing what type of answer we receive. Like the Ephraimites in the story it turns out it might be our very tongues that keep us from getting what we desire — accurate data.

Connect with us on our social media platforms: Twitter, Facebook, Instagram, LinkedIn and YouTube.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Busara Center

Busara Center

379 Followers

Busara is a research and advisory firm dedicated to advancing Behavioral Science in the Global South