Boy, Girl or Chatbot?
Mitigating Gender Bias in Artificial Intelligence
When you think of a virtual assistant the names Siri, Alexa and Cortana immediately spring to mind. But have you ever stopped to think about why some of the most popular chatbots have been assigned what might typically be considered to be female names? Is this mere coincidence or something we should be paying closer attention to?
Chatbots, of course, are developed by and for humans, and humans carry their own thoughts, feelings, perceptions and biases (both consciously and unconsciously) into the decisions they make. Ask many people to think of someone as an assistant, secretary or clerk, and it’s likely that a lot of people will picture a female in the role as we each bring our own preconceptions, perhaps shaped by our own cultural experiences, to bear. With Siri, Alexa and Cortana and the rest, have we now extrapolated our preconceptions to the world of chatbots? And why does this matter?
When building systems with artificial intelligence (AI), it’s a natural desire to assign human characteristics. Human interaction, after all, is what humans have most experience of, so it makes sense to try and mimic this kind of interaction. But are designers and developers perpetuating some outdated undesirable stereotypes in this emerging field and what, if anything can be done?
Throughout my career I’ve designed chatbots and systems for many different companies and many different applications. Reviewing conversational logs, I’ve often been shocked by the language people use when interacting with a bot. You don’t have to dig too far to see insults, sexism, and general mistreatment, of the kind that humans would find hard to tolerate. Unfortunately, my own experience suggests that more offensive language can be found in conversation logs for chatbots characterised as female, and less for those characterised as male or neutral. So, as chatbot designers do we have an ethical duty to temper misogynistic, homophobic and racist interactions that are currently littering the chatlogs? And, if so, how do we do this?
Sage’s assistant, Pegg, was deliberately designed to be gender-neutral. This creates challenges for conversation design, because languages were designed to be spoken by humans. When designing conversations for Pegg we need to pay special attention to avoid any cultural reference to gender (either consciously or unconsciously) and this can be hard. How should Pegg respond to questions like “Do you have a boyfriend?”, “Are you a boy or a girl?” or “Do you want to date me?”. This problem is particularly acute for languages in which gender is grammatically more prominent, such as those with a Latin origin (sometimes referred to as the Romance languages). It is sometimes difficult to avoid any reference to gender in answers, because the majority of grammatical (and many lexical) words are gender dependent. This being said, there are things you can do to mitigate this; reducing the occasions when your bot talks about itself, using generic words instead of those perceived as gender-specific, and avoiding using gender-specific words in favour of generic statements.
Chatbots integrated with voice present further challenges, as a human-like voice for a gender-neutral bot can make users feel uneasy — a phenomenon that’s been described as the ‘uncanny valley’[1]. To counter the problem there have been attempts to create synthetic voices — voices that sound natural but are difficult to associate with any specific gender — such as Q, described as ‘the first genderless voice for assistants’[2].
Of course, it’s not just how our bots sound, it’s what they say (and in response to what) that’s important too. With training data often taken from real material, where the language we use is somewhat skewed, our bots can also inherit this bias. When training a chatbot its algorithm pays close attention word embeddings — combinations of words that frequently come together — and what semantic and syntactic relationships come together to convey meaning and intent. Critics say this approach perpetuates gender bias — tying gender-neutral words to particular genders — ‘boss’ being associated with ‘he’ or ‘receptionist’ to ‘she’[3]. There have been attempts[4] [5] to detect and remove this unintended bias from the training data in AI systems but there’s clearly a way to go yet.
There are ethical questions too when it comes to the influence and impact that bots have on those who use them. Bots often have a tendency to agree with a user, particularly when they don’t understand the full utterance. This can give rise to a user who states they ‘love’, ‘adore’ or ‘like’ some quite questionable things and the bot greeting such statements positively.
As chatbot designers we need to understand the influence our bots have and to take our ethical responsibilities seriously — disagreeing with the user where it makes sense to do so. As bot interactions increase we should remember too that they’re a very visible manifestation of a brand’s voice and values. Making sure bot conversations accurately align with desired brand perceptions clearly needs careful planning. As Maaike Groenewege puts it, in the near future, we might want to turn to a chatbot’s persona as a base for laying out its ethics, transparency and explainability as well. A persona as a set of ethical validation rules for all its content[6].
Google Assistant serves an interesting case study. The user isn’t so much interacting with a male or female but with the brand itself. This significantly reduces the ‘Hey Google, are you a boy or a girl?’ type questions as the whole experience is framed as a brand interaction. This approach, of course, brings its own challenges — it can be hard to make the interactions feel human-like and evoke emotion, and there’s danger the whole brand can be tarnished by poor interactions. Google’s approach also provides some clarity to avoid the situation where users are not always aware they’re talking with an automated system and not a real person.
Tech companies are not usually shy when it comes to shouting about their inclusivity and ethical approaches but it’s clear in the emerging field of artificial intelligence there’s more that we all could do. Datasets should be more diverse and inclusive, and strategies should be incorporated into the development pipeline for detecting and removing unintended bias. The key is awareness, inherited biases are not always easy to spot, but they can be certainly mitigated if we put in the effort to achieve it[7].
References:
[1] Lay, S. “Introducing the Uncanny Valley”, https://www.uncanny-valley.co.uk/?OpenForm
[2] “Meet Q The first genderless voice” https://www.genderlessvoice.com/
[3] Tatman, R. ”Gender bias in Word embeddings?”, https://www.kaggle.com/rtatman/gender-bias-in-word-embeddings
[4] Wiggers, K. “Salesforce researches claim new method mitigates AI models’ gender bias”, https://venturebeat.com/2020/07/01/salesforce-researchers-claim-new-method-mitigates-ai-models-gender-bias/
[5] Lu, K.; Mardziel, P.; Wu, F.; & Amancharla, P. “Gender Bias in Neural Natural Language Processing”, https://arxiv.org/pdf/1807.11714.pdf
[6] Groenewege, M. “Blender turned rogue?”, https://chatbotslife.com/blender-turned-rogue-f3c0a64edc5a
[7] Yao, M. “Bigoted bots: Racial and Gender bias in Artificial Intelligence”, https://www.topbots.com/bigoted-bots-chatbots-racial-gender-bias-artificial-intelligence/
