ChatGPT, an artificial intelligence chatbot that generates eerily human-sounding text responses, represents the new and advanced face of the debate over AI’s potential — and dangers. The technology has the potential to assist people with everyday writing and speaking tasks, but some are concerned because the chatbot has been known to allow users to cheat and plagiarize, potentially spread misinformation, and enable unethical business practices. What’s more, it’s rife with bias, just like many previous chat bots.
OpenAI, the company behind the first GPT and its subsequent versions, added guardrails to ChatGPT to help it avoid problematic responses from users who, for example, asked the chatbot to say a slur or commit a crime. Users, on the other hand, discovered that rephrasing their questions or simply asking the program to ignore its guardrails resulted in responses with questionable — and sometimes outright discriminatory — language.
For example, Galactica was an LLM similar to ChatGPT trained on 46 million text examples that was shut down by Meta after 3 days because it spewed false and racist information.
As the world becomes more reliant on technology, AI is being used to make critical decisions in fields such as policing and healthcare. However, biased algorithms mean that existing inequalities may be amplified — potentially with disastrous consequences. Thus, it is important to be aware of its bias because we can see that his answer to this topic is not satisfying!
Learning from a biased society
The first bias observed is the prejudice that the Language Model (LM) inherently possesses because of human bias. Indeed, Chat GPT receives data from websites such as Reddit and Twitter. As a result, it is more likely that the voices of those who support dominant points of view will be preserved. White supremacist, sexist, ageist, and other points of view are overrepresented in US and UK English training data, which not only outnumbers their prevalence in the general population but also sets up models trained on these datasets to amplify biases and harms.
The output from Chat-GPT confirms human general bias and sexism that we have all encountered in our daily lives and workplace.
Starting with who contributes to these Internet text collections, we learn that Internet access is not evenly distributed, resulting in Internet data that favors younger users and those from developed countries. Nonetheless, specific subsamples of the Internet are being scrutinized rather than the Internet as a whole. Reddit’s outbound links, for example, are used to scrape the training data for GPT-2; according to a Pew Research survey, 67% of Reddit users in the United States are male, and 64% are between the ages of 18 and 29. According to comparable data, only 8.8–15% of Wikipedians are women or girls, according to recent surveys. Thus, because it is inherited by humans, the first pattern of bias detected on Chat GPT is the initial participation on the internet.
Another source of bias is that even though user-generated content websites such as Reddit, Twitter, and Wikipedia advertise themselves as open and accessible to all, they are less welcoming to minority populations due to structural issues such as moderation processes. There have been numerous instances on Twitter where users who receive death threats have their accounts suspended while the accounts issuing the threats continue to operate. Finally, this systemic pattern exacerbates diversity and inclusion in Internet-based communication, creating a feedback loop that reduces the impact of data from underrepresented populations. A small number of subpopulations can continue to easily add data, share their thoughts, and build platforms that reflect their worldviews.
Then there is bias in the filtering step; indeed, the current practice of censoring datasets can further muffle the voices of those who identify as marginalized. The GPT-3 training set was a filtered version of the Common Crawl dataset created by training a classifier to identify documents that were most similar to those used in GPT’s training data, which included documents linked to from Reddit, Wikipedia, and a selection of books. While it is claimed that this was successful in removing papers labeled as “unintelligible” by previous research, it is unclear what else it apparently removed. The Colossal Clean Crawled Corpus, which is used to train a trillion parameter LM, is cleaned by removing any page that contains one of roughly 400 “Dirty, Nasty, Obscene…” words. By stifling terminology like “twink,” this will undoubtedly weaken the power of internet forums created by and for LGBTQ people. Filtering out the language of underrepresented communities results in a lack of training data that reclaims insults and generally portrays marginalized identities favorably.
As a result, current practice favors the hegemonic viewpoint at every stage, from initial engagement to ongoing participation via the collection and, eventually, filtering of training data. Accepting a large amount of web content as “representative” of “all” of humanity jeopardizes dominant ideologies, increases power differences, and solidifies inequality.
Locked in the past
Since data is collected from the past, Chat GPT tends to have a regressive bias that fails to reflect the progress of social movements. The deliberate use of language to undermine prevailing narratives and draw attention to social perspectives that are underrepresented is a key component of social movement building. New social conventions, languages, and communication channels are created by social movements. This makes it more difficult to apply LMs since approaches dependent on LMs suffer the risk of “value-lock,” in which LM-dependent technology solidifies earlier, less comprehensive understandings.
For example, as the Black Lives Matter (BLM) movement progressed, it had an impact on the creation and modification of Wikipedia entries, increasing the coverage and decreasing the latency of articles about shootings of Black people. Furthermore, as new entries were published, articles documenting previous shootings and instances of police brutality were updated, demonstrating how social movements connect events over time to develop coherent narratives.
A critical caveat is that social movements with insufficient documentation and little media attention will not be covered at all. Events that challenge state power, as well as social movements and protests, may not receive adequate media coverage. One example is media outlets that frequently disregard peaceful protest activities in favor of dramatic or violent events that make for compelling television but almost never receive positive publicity. As a result, data used to support LMs risks underrepresenting social movements and overly aligning with current power structures.
If the training data is not frequently updated, developing and shifting frames may be learned inadvertently or become lost in the massive data used to train massive LMs. Even large organizations may be unable to afford the computing costs associated with training large LMs on a regular enough basis to keep up with the type of linguistic change described here. Perhaps fine-tuning methods could be used to retrain LMs, but careful curation practices would be required in this case as well to identify the right data to capture reframings and methods to assess whether such methods accurately capture the ways in which new framings challenge hegemonic representations.
Should ChatGPT be woke?
Finally, we can say that Chat GPT is a complex model to judge, it is both very useful, accessible and easy to use, but it also has many significant biases, so its use requires great caution. The fact that we cannot have a model that is not political is the greatest proof of this. Either one takes the data as it exists and the model becomes conservative, in effect accepting the state of the world as it is, which is political, or one decides to commit oneself and therefore one would make the model give more voice to minorities (more weight to minority data), which is also political.
Bibliography
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell
Inherent Human-bias in Chat-GPT, Rahul Bhadani
Moral Codes, Alan Blackwell