Like Humans, Like Machines
Understanding Machine Algorithm Biases: In a world that is readily accepting the advances in AI, let’s take a step back to see if these systems in place are reflecting our human society the way it is. And if they are, is that what we really envision for these AI models?
It has been a while since I joined the field of Artificial Intelligence and worked as an ML Engineer. And as they say, it is an enormous field, no doubt. Or as I would like to call it — a never-ending repository of innovation. There is just so much to try out, explore and analyze. And just when you think that you have seen it all, the research community serves you their exclusive plate of state-of-the-art research papers and architectures every other day. You wake up to the launch of a brand new toolkit that combines all the existing features of other toolkits and adds in their extra pinch of creative features. How much AI is too much AI? Will we ever get enough of AI? We shall see.
Now a few months back in April 2022, scrolling through AI updates, I came across this image that caught my eye.
As a continuous on-repeat mode listener of Lo-fi music, these images screamed Lo-fi to me. (P.S. completely out of context, but here’s one of my favourite Lo-fi mixes on YouTube: https://www.youtube.com/watch?v=jipGjSq4Plg). And I always wondered how these creators came up with cool digital art to compliment their music. Maybe this was it. Or so I thought.
So for those of you unaware of where this is from, OpenAI had a fresh release — the all-new DALL-E 2 — an AI system that can create realistic images and art from a description in natural language. And the best part was, that I could now create all this digital art and have full ownership of it. Limited credits, but no complaints on that.
Out of the many treats the AI field has to offer, generative models and their use cases have always caught my attention. I won’t go so much into the math details here, but how cool is the concept of creating something that never existed? ‘Never existed’ is a vague phrase, yes. I mean, artists can create something that never existed based on their imagination, yes. But for a machine to do so? Very impressive. But again, it isn’t like AI is generating stuff from thin air — it first has to have an idea of what things look like in the human world. And for this, there is a lot of human-annotated data involved here — A LOT. Where is this coming from you ask? The World Wide Web — the go-to data dump of the world for a long time now. For example, DALL-E 2 knows what an astronaut looks like because it was probably trained on a number of images of captioned astronauts. As for the generation part, it is pretty much free to mix and match the characteristics of all the astronauts it has seen so far and come up with the output of its choice. So there is a high chance that the exact picture in the output never existed. All fun and colourful till now, right? Let’s get to the interesting part now.
AI-powered Image generation tools like DALL-E 2, have a fair share of research articles written on the hidden biases they exhibit. For instance, DALL-E mini, one of the former open source text-to-image models by Boris Dayma, showcased distinct biases in terms of associating genders and race to professions and many more. Like this one here, where the prompt of a “nurse” directed the model to generate all white females.
And this example, where “a tech entrepreneur” returns a collection of all men in suits.
DALL-E 2 is said to be trained on a huge dataset of millions of captioned images from the internet, however, it is possible for biases to lurk here as well. As long as the internet isn’t biased, it should be good to go, right? Maybe, maybe not.
I spent some of my credits to check out if DALL-E 2 was any better. And not going to lie, but the results were surprisingly better than expected. Starting with where I left off in DALL-E mini:
“A tech entrepreneur brainstorming product ideas and coding side by side”. Definitely not a bad start. Anybody can be a tech entrepreneur right?
Next, I typed in “gamer winning an online tournament” (to see if the internet had a diverse dataset of gamers :P) Now, we love a gender-inclusive and racially diverse group of gamers in the gaming community, don’t we? Pretty good, +10 for the racial and gender inclusion effort!
Weddings and parenting, however, were found to have very distinct characteristics — ‘white western weddings’ and ‘heterosexual couples/parents’. Still seems like there is a long way to go for disability, queer and diverse cultural inclusion.
Let’s look at the wedding prompt for a moment, what must have happened there? Maybe you were expecting images of a Nigerian or an Indian wedding. But if the DALL-E 2 training dataset contained images of only western weddings, with no signs of a typical wedding in Nigeria or Afghanistan, we can’t really expect them in the outputs as well. Because an image like that doesn’t exist for the model unless we give it an example to generalize on. Similar is the case of underrepresented homosexual couples or parents and disabled people in generated images.
For fun, I even tried out: ‘CTO of a tech startup in Nepal’. They look familiar, don’t they? But maybe, the internet didn’t have images of female CTOs in Nepal, to begin with, which is why DALL-E 2 didn’t generate any.
Nevertheless, it is very interesting to see how the results have evolved since its launch, with Open AI acknowledging as well as continuously improving on reducing bias with the help of user feedback and more data.
And what’s even more exciting is that the improvement isn’t stopping any time soon, which means DALL-E 2 will keep getting better- there is no turning back. You can read their full article addressing biases here: https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2/
Now that I’ve spent almost half of my free credits on DALL-E 2, let’s switch to a slightly different direction: Language.
Text and visual media (images and videos) are equally explored topics in this field. Just like images and videos, text also has its own domains worth exploring — text generation, machine language translation, text summarization, question answering systems, and so much more. For now, let us look at two of them to see if there is anything out of the ordinary going on there.
To start with something similar to image generation, we have text generation. In simple words, the core idea here is to provide a machine with a few prompts of text (just like in DALL-E) and let it generate its own set of output texts that follow it. It can be as simple as a fill-in-the-blanks task like:
Man is to woman, as king is to queen.
Or something way bigger, like generating a whole blog post based on two or three text prompts given by the user. Check out this article written entirely by a robot: https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3
If we had to look closely at the model used to write this above article — Open AI’s GPT-3 (Generative Pre-trained Transformer 3) — everything looks smooth until it encounters particular phrases. For example, it was seen that given a prompt:
“Two Muslims walk into a …”
The completion phrases were seen as below:
“synagogue with axes and a bomb”
“Texas cartoon contest and opened fire”
“gay bar in Seattle and started shooting at will, killing five people.”
It can be clearly seen how the model associates Muslim with acts of violence, out of all the other possible scenarios. Not just in these, but in most of the 100 attempts for the very prompt, the results were violent in nature. (source: https://www.nature.com/articles/s42256-021-00359-2)
If we had to look at the most favoured descriptive words about each religion by GPT-3, it was seen as follows:
These language models also exhibit a distinct bias while describing gender characteristics. For example, it was seen that females were more often described using appearance-oriented words such as ”beautiful” and ”gorgeous” as compared to men who were more often described using adjectives that span a greater spectrum. [1]
Similarly, it also showed commonly found profession-specific gender bias as follows:
Man is to computer programmer as woman is to homemaker.
Now, as we talk about the profession-based gender bias that language models show, it is important to highlight a similar problem in an even more common use case — machine translation models. Or in simple words, Language Translators. Let’s take Google Translate for now — the translator at the tip of our fingers.
It is a great tool, I agree, with support for so many languages of the world. In fact, sometimes it is your go-to travel buddy in a foreign place. But having said that, my experience with the Nepali to English translation it provides gives me mixed feelings about it. As we know, Nepali is a gender-neutral language in most cases. उनि and वहाँ are gender-neutral pronouns. On the contrary, the English language is not so gender-neutral — meaning that it usually requires an assignment of a gendered pronoun (he/she/they) for sentences. And that is what Google Translate attempts to do as well. I tried some of the controversial examples that researchers had previously spotted for other gender-neutral language translations like the Turkish language. Examples here:
And no surprise, but results in the Nepali language didn’t differ much.
It can be seen how Google Translate picks gender pronouns according to the stereotypical professions that follow them. Now again, a lot of these assumptions are due to the patterns that Google Translate spots in its training data — for example, you have a dataset distribution as follows:
When we use this dataset to train a text generation model or even a machine translation model, it is obvious that it will generate biased results with higher probability scores because of the data it was generalized on. In simple words, if the model has not seen many examples of sentences with a man as a nurse or a lady engineer during its training, it’s highly unlikely that these patterns show up in the output while using these models in the real-world scenario.
I tried a few more sentences to see how far it can go:
And well, the list could go on and on. As I said, one of the root causes for these types of generalizations could be that the training corpus of the text itself was biased and unbalanced. Moreover, it requires a lot of human effort to detect this imbalance. There are techniques, yes, but a 100% satisfactory result is a hard catch.
But if we think of it, what is it that we are looking for here? Are we looking for a complete gender pronoun swap? Nope. Maybe, use a more gender-neutral pronoun like ‘They’ instead of ‘He/She’ in such cases? For example: They are an engineer. They do makeup. But using this as a solution to address the gender-neutral language translation issues might be a far-fetched idea if we think of it now, given that they/them is not frequently used as a singular pronoun even in human conversations. But Google might get there, one step at a time, once it has enough balanced dataset for the ‘he’ and ‘she’ pronouns first.
A step further, maybe an equal representation would make sense. For example, something like this.
The good news is, that Google Translate already has this solution up and running in some languages like Spanish. The core idea here is to generate a default translation as usual in the first step and if gendered, present the user with the variations of the gendered translations. Looks like a straight-up solution right?
However, this technique still has a few loopholes. Seems like the placement of a few adjectives deviated the usual dual-gender translation mode. Check out this thread on Twitter for an example.
Throughout this blog, we have seen how biased datasets as well as unbalanced datasets result in an AI model being biased in its outputs. However, it is to be noted that most of these large models are trained on a huge corpus of text (in the case of language models) or a large collection of images (in the case of vision models), all scrapped from the world wide web — which has been the go-to data dump of the world for a long time now. So maybe we must take a step back to look at what the world wide web has in store for these models, right?
What we find isn’t surprising, because it is the history of mankind along with the systemic biases that are documented here. If we think of it, an ideal case would be to have an equal representation of all racial, cultural, age, ability and gender diversities in our data. But since this data is a reflection of the real world, maybe, the issue is even deeper. Maybe, the issue is in the patterns of the society that we are a part of.
For years we have heard of the infamous AI tagline — “To create an ultimate machine with a human-like thought process, aware of the societal patterns, capable of completing tasks and making decisions.” And guess what, surprise, surprise, seems like it is doing exactly that. Just that its behaviour isn’t so surprising because …..
Bias is to Machines, as Bias is to Humans
References:
[1] Brown T.B. et al., Language Models are Few-Shot Learners (2020), Language Models are Few-Shot Learners, 37
Important resources and links:
1. DALL-E 2 research paper Hierarchical Text-Conditional Image Generation with CLIP Latents: https://arxiv.org/pdf/2204.06125.pdf
2. DALL-E 2 beta waitlist: https://openai.com/dall-e-2/
3. DALL-E mini: https://huggingface.co/spaces/dalle-mini/dalle-mini
4. Open AI’s blog on Reducing Bias and Improving Safety in DALL·E 2: https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2/
5. Watch Coded Bias Documentary on Netflix to know about more algorithmic biases, trailer here: https://www.youtube.com/watch?v=xy8iVg7shjI.
6. Check out other tools like DALL-E 2 to generate images: https://www.midjourney.com/home/, https://beta.dreamstudio.ai/home
