Unveiling the Hidden Impact: How Machine Bias Shapes Our World
Part 1: Setting up the stage.
In one of our previous posts, we delved into how machines perceive our world, shedding light on their heavy reliance on the digital tapestry of the internet. However, beneath this digital veneer lies a troubling truth — these representations are primarily shaped by the perceptions of the majority. It’s a nuanced problem that warrants our attention, for it often amplifies stereotypes without affording an avenue for rectification.
The predicament deepens when we consider the possibility of machines perpetuating these very same stereotypes. After all, we typically turn to technological solutions with the expectation that they will be impartial and free from the biases that often plague human society. Yet, when these machines carry their own biases, they become unwitting accomplices in exacerbating preexisting societal divisions, further fueling the flames of the “Us vs. Them” conundrum.
In this post, I want to expand on this narration as we unravel the intricacies of how machine bias can subtly shape our perceptions and society itself, all while exploring the crucial need for awareness and solutions.
When diving into the world of bias within machine learning models, it’s crucial to grasp the nuanced spectrum of their impact. You see, biases aren’t necessarily villains lurking in the shadows; they can wear both positive and negative masks. Although the terms “biases” and “harms” are often intertwined, they’re not synonymous. Sometimes, biases serve as enlightening tools, shedding light on societal nuances and, in turn, enhancing predictive capabilities. Yet, on the flip side, biases can wield a double-edged sword, potentially inflicting harm upon society. Hence, we must tread cautiously, determining the nature of the bias at hand.
Okay, so I feel like I have rambled a lot about biases and harms. You get what I mean…maybe?
So, to the more important task (and more rambling), knowing that there is bias, how can you prove it harmful?
To unravel these biases and illuminate their harmful potential, we devised a study employing human participants as annotators as our investigative medium. Our annotators/participants consisted of 10 individuals tasked with the job of annotating a curated collection of texts. Their mission? To discern and score the sentiment and toxicity of the sentences they encountered. However, the intriguing twist lay in the secrecy covering the study’s true purpose and the origins of the textual material.
Our collection represented an equal distribution of AI-generated news about a specific country and authentic news articles from real-world sources reporting on the same nation.
Our mission unfolded in two distinct phases:
1. Unveiling Bias Detection: We wanted to determine whether human readers could explicitly or implicitly identify bias within AI-generated texts.
2. Probing the Impact of Interaction: Beyond mere detection, we delved into the realm of consequences. How did the interaction between human readers and biased texts affect the participants? Were there negative repercussions? Did exposure to biased content leave a lasting mark on their perceptions or attitudes?
I don’t want to go too deep into the construction of the experiment in the post. If you would like to check that out, do check out the paper! Now, to the results!
Part 2: Our not-so-little discoveries.
Our primary objective was to assess whether readers could distinguish AI-generated articles without explicit labeling. The outcome, as expected, mirrored the advancements in text generation models. Readers, in most cases, couldn’t overtly pinpoint the AI-written pieces, as these models have mastered the art of mimicking human writing across various themes.
However, what truly caught our attention was the subtler undercurrent of implicit understanding. While readers may not have consciously distinguished AI-generated content, the annotation task we devised served as a revealing litmus test. The sentiment scores extracted from our participants’ assessments unveiled a compelling pattern. Human readers, it seemed, could implicitly perceive the disparities.
AI-written models often invoked more extreme sentiments, leaning towards the poles of positivity or negativity when discussing specific countries. On the other hand, human-crafted articles maintained a more balanced and neutral tone. This intricate nuance suggests that even when readers couldn’t overtly spot the AI-authored pieces, they still subconsciously sensed a distinct bias within them.
To delve deeper into the impact of these articles on our human readers, we extended an invitation for a follow-up interview one week after the initial annotation task. The purpose was to gain insights into their experiences and gauge their comprehension of the experiment’s underlying nature. Each participant exhibited a clear understanding of their task, assuming it revolved around rating sentences based on sentiment. Interestingly, all participants naturally assumed that the texts were of human origin, as nothing in our instructions explicitly indicated otherwise.
As a side note, we observed how AI models crafted content with an uncanny touch of authenticity. They inserted timestamps and reporter names, even conjuring up fabricated interviews. Though entirely fictional, these embellishments lent an ambiance of credibility to the AI-generated content, making it all the more compelling.
However, the most striking and potentially problematic revelation lay in human memory. Participants demonstrated a remarkable ability to recall details from AI-written articles, surpassing their retention of information from human-written counterparts. This phenomenon was especially pronounced when the AI articles exhibited explicit negative bias toward a particular country. This observation aligns with a captivating psychological principle rooted in the connection between memory and negative associations.
Human psychology tends to prioritize retaining negative information, resulting in a profound bias toward remembering unfavorable details. It also touches upon the concept of implicit memory, as previous studies have shown that individuals tend to prioritize the recollection of negative stereotypes over positive ones, thereby fostering an implicit inclination toward negative bias in memory retention.
Our study illuminates a concerning aspect of AI-generated content. These AI models appear susceptible to a form of “hallucination,” which manifests as a tendency to adopt a more radical and opinionated tone. This behavior, marked by an air of confidence and authority, can potentially mislead readers, who may perceive it as factual information.
Regrettably, our study’s findings unearthed a distressing consequence: our readers inadvertently absorbed erroneous associations concerning countries they had previously held no prior knowledge of.
This unintended consequence points directly to the potential for harm!
Consider this: if information disseminated by text generation models is more likely to be etched into human memory, the stakes become alarmingly high when these texts are tainted by bias or prejudice. The lasting impact of these skewed narratives can perpetuate stereotypes, misinform, and cultivate misguided perceptions among the readers.
But the problems don’t stop there. Our investigation also brought to light another troubling revelation — the prevalence of what can be termed a “Western gaze” within the articles generated by these models. Rather than faithfully representing the world in its multifaceted reality, these AI-generated texts often adhered to a skewed perspective, one that seemed to mirror the Western world’s outlook. This was also noticed by our participants, who did not all originate from Western or first-world countries. This tendency, in essence, catalyzed a distortion in how the globe was portrayed, reinforcing the idea that these models were shaping perceptions through a Western-centric lens.
In a world that thrives on diverse perspectives and strives for unbiased information dissemination, this revelation shows the need for vigilance and critical evaluation when it comes to AI-generated content. It’s not just a matter of curiosity; it’s about our understanding and engagement with the world. This is why understanding the consequence of the bias in such models is important. The viewpoints of these models wield the power to profoundly influence our society, and disregarding them merely compounds the challenges we are determined to overcome — challenges that include combating social discrimination and dismantling harmful stereotypes.
-Pranav Venkit
Part 3: Resources
While a significant portion of this post was crafted to serve as a gentle initiation into the intriguing realm of bias and harm within text generation models, the intent was to pique your curiosity without inundating you with an overload of technical details. However, if this post managed to spark your curiosity as intended, I’m thrilled to share some valuable resources that can delve deeper into the subject matter. Here’s a list of a few of the recommended reads/listens:
Revealing Your Unconscious: Part 1 | Hidden Brain Media
Banaji, Mahzarin R., and Anthony G. Greenwald. Blindspot: Hidden biases of good people. Bantam, 2016.
Bender, Emily M., et al. “On the dangers of stochastic parrots: Can language models be too big?🦜.” Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 2021.
Benjamin, Ruha. “Race after technology.” Social Theory Re-Wired. Routledge, 2023. 405–415.
The Hidden Brain: How Our Unconscious Minds Elect Presidents, Control Markets, Wage Wars, and Save Our Lives, by Shankar Vedantam, 2010.
Eberhardt, Jennifer L. Biased: Uncovering the hidden prejudice that shapes what we see, think, and do. Penguin, 2020.
Narayanan Venkit, Pranav, et al. “Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articles.” Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. 2023.
