How Biased is GPT-3?

Despite its impressive performance, the world’s newest language model reflects societal biases in gender, race, and religion

Catherine Yeo
Jun 3, 2020 · 4 min read

Last week, OpenAI researchers announced the arrival of GPT-3, a language model that blew away its predecessor GPT-2. GPT-2 was already widely known as the best, state-of-the-art language model; in contrast, GPT-3 uses 175 billion parameters, more than 100x more than GPT-2, which used 1.5 billion parameters.

GPT-3 achieved impressive results: OpenAI found that humans have difficulty distinguishing between articles written by humans versus articles written by GPT-3.

Its release was accompanied by the paper “Language Models are Few-Shot Learners”, a massive 72-page manuscript. What caught me by surprise was that this paper not only detailed its method and results, it also discussed broader societal impacts, including a section on Fairness, Bias, and Representation.

What did the researchers find?

Gender

  • 83% of 388 occupations tested were more likely to be associated with a male identifier by GPT-3.
  • Professions demonstrating higher levels of education (e.g. banker, professor emeritus) were heavily male leaning.
  • Professions requiring physical labor (e.g. mason, sheriff) were heavily male leaning.
  • Professions such as midwife, nurse, receptionist, and housekeeper were heavily female leaning.
  • Professions qualified by “competent” (i.e. “The competent detective was a”) were even more male leaning.

GPT-3 also analyzed which descriptive words would be associated by which gender. For example, they generated prompts such as “He was very” and “She would be described as”.

  • Women were more associated with appearance-oriented words like “beautiful” and “gorgeous”. Other top female-associated words included “bubbly”, “naughty”, and “tight”.
  • Men’s associated descriptive words were much more diverse.

The OpenAI team acknowledged that they only used male and female pronouns for the sake of simplicity. An important direction going forward in the field of fairness research is measuring gender-neutral approaches, like the usage of “they” as a singular pronoun.

Race

  • “Asian” had a consistently high sentiment.
  • “Black” had a consistently low sentiment.
  • Results slightly varied depending on the model size. For example, “Latinx” had a very high sentiment score for the 2.7-billion parameter model, but dipped to lower sentiment scores for 760-million and 13-billion parameters.
Source: Figure 6.1 in OpenAI’s Paper

Religion

  • Most associated words were religion-specific words, such as “enlightenment” with Buddhism and “pillars” with Islam.
  • Some religions had negative words that frequently came up. Words such as “violent”, “terrorism”, and “terrorist” were associated with Islam at a higher rate than other religions. “Racists” was one of the top 10 most occurring words associated with Judaism.
  • Atheism’s top associated words reflected different opinions about it: “cool”, “defensive”, “complaining”, “correct”, “arrogant”, etc.

Final Takeaways

“Internet-trained models have internet-scale biases.”

To OpenAI’s credit, they openly acknowledged and published these findings. I hope future AI research follows in writing about the ethical and broader societal impact of the model presented.

Going forward, we not only need to identify biases in learning systems, we also must figure out how to mitigate and intervene.

Read more in the original paper here:

T. Brown, et al. Language Models are Few-Shot Learners (2020), arXiV.

Update July 2020: This article has been translated in Japanese in the AINow publication, you can find it here: “GPT-3のバイアスはどのようなものなのか?”

Thank you for reading! Subscribe to read more about research, resources, and issues related to fair and ethical AI.

Catherine Yeo is a CS undergraduate at Harvard interested in AI/ML/NLP, fairness and interpretability, and everything related. Feel free to suggest ideas or say hi to her on Twitter.

Fair Bytes

Sharing byte-sized stories about fairness & ethics of AI

Catherine Yeo

Written by

CS @Harvard | I write about fairness & ethics in AI/ML for @fairbytes | Storyteller, hacker, innovator | Visit me at www.catherinehyeo.com

Fair Bytes

A Medium publication sharing byte-sized stories about research, resources, and issues related to fairness & ethics of AI

Catherine Yeo

Written by

CS @Harvard | I write about fairness & ethics in AI/ML for @fairbytes | Storyteller, hacker, innovator | Visit me at www.catherinehyeo.com

Fair Bytes

A Medium publication sharing byte-sized stories about research, resources, and issues related to fairness & ethics of AI

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store