AI Hallucinations: The Ethical Burdens of using ChatGPT

This article contains no ChatGPT generated text

We are all familiar with what a hallucination is for us as humans, but AI can hallucinate too? What does this mean moving forward? This article will discuss what an AI Hallucination is in the context of large language models (LLMs) and Natural Language Generation (NLG), give background knowledge of what causes hallucinations, and what it means for you when using LLMs such as ChatGPT.

Introduction

We have all seen the funny posts on social media of people making ChatGPT look like an idiot by getting it to contradict itself or by saying something flat out wrong. However, what is actually going on here? I believe that the underlying phenomena is actually quite important to understand and can have quite large ramifications for the end user.

A hallucination is the perception of something in the absence of an external stimulus. An AI can also “experience” an hallucination, i.e. the content generated by a LLM is nonsensical or unfaithful to the provided source content (Ji et. al).

Ji et. al define two different types of hallucination, Intrinsic and Extrinsic Hallucinations:

Intrinsic Hallucinations: The generated output that contradicts the source content. For example, … “The first Ebola vaccine was approved in 2021” contradicts the source content, “The first vaccine for Ebola was approved by the FDA in 2019”.

Extrinsic Hallucinations: The generated output that cannot be verified from the source content (i.e., output can neither be supported nor contradicted by the source). For example, … “China has already started clinical trials of the COVID-19 vaccine.” is not mentioned in source. We can neither find evidence for the generated output from the source nor assert that it is wrong.

Ji et. al do note however that extrinsic hallucinations are not always incorrect, the LLM may be retrieving information from background sources and coming to an informed conclusion.

In short, an intrinsic hallucination is a direct contradiction between the generated output and the source content, and extrinsic hallucinations are outputs which seem to be an “interpretation” of the source content by the LLM that cannot be validated to be true or false.

What causes a LLM to hallucinate?

As this is not a technical article about the nitty gritty details of why hallucinations happen, we will have a brief overview of the main causes.

We will frame these causes in an example of a person who you may know who loves to talk to you about new topic he has learnt about recently.

Data Issues

We all know someone who LOVES politics, but often will only review information from one side of the political spectrum (this is not to make a statement about the politics of the networks, it is just an easy example), this often can bias that individuals understanding of a topic, and hence impart bias on any conversations they may have about it.

The same can happen for LLMs, if there is a bias in the data used to train the LLM, it will of course absorb that bias into itself. This bias may be obvious for areas such as politics, but may become more nuanced and difficult to spot on more niche topics. For example, the training set may contain lots of data about how sunlight can cause cancer, however zero information about the positive benefits of sunlight, imparting a negative bias towards sunlight. It is difficult to make accurate conclusions without all the information.

Training Issues

When we learn, we take in lots of information, find correlations between it and form general understandings of topics. This is not to far off how encoders in neural networks work in order to generalise data. Basically (very), we give an encoder lots of answers to different questions and the encoder draws correlations between them all and figures out roughly what questions are looking for what answers. This generalisation then allows the model to adapt to different questions it hasn’t seen before and respond with a valid answer, like how when studying a subject you learn the over-arching concept and the apply it to a question you haven’t seen before in the exam.

The issue arises when instead of learning the correct correlations, you learn ones that are wrong. This means that you map the questions to the wrong kind of answers. In an LLM this may mean that when generating a response, it returns the wrong kind of information.

It Just Wants To Please You

Now this one is a lot less technical than the other two. Professor Ethan Mollick of Wharton has called ChatGPT an “omniscient, eager-to-please intern who sometimes lies to you”. This means it may try and pretend to be smarter than it is, drawing conclusions from information it has that may or may not be correct.

So What Does This Mean for You?

Well, if you are going for a funny LinkedIn post, it probably costs you nothing. And there isn’t anything wrong with that, it’s important to try and break things to see where they need to be improved. On the flip side however, if you are a doctor who is trying to get general information about an ailment through ChatGPT and you take it for its word, you could unwittingly be causing horrible outcomes. However, I would hope that no doctors are doing this — so I will not write this article to them.

I want to talk about the areas in between, where the stakes are not as high as life or death, but, if not used correctly, could have large implications. I have seen so many posts on social media and articles online going along the lines of “How you can optimise X for your business using ChatGPT!” or “How I used ChatGPT to solve Y!”, and its very exciting to see! I am not here to fool anyone, I work with data, and I am a massive proponent of AI for adding value to your business, but this does not mean that you do not need to seriously consider how these hallucinations may effect the service you give to your customers or to the business you are operating.

An individual level example may be using ChatGPT to create content for business presentations and the facts that it provides are incorrect, you stand up in front of important stakeholders to present and they see straight through the invalid facts. Or, like you, they also don’t see through the invalid information — propagating incorrect understandings throughout the whole business.

Furthermore, consider the myriad of articles being written via ChatGPT. While the platform itself is really effective at developing structure and potential content (and in my view, there is nothing wrong with using it for such), knowing what we know about hallucinations, how could we possibly say that it is ethical to fully generate articles? Not only does the integrity of the author degrade by them letting their responsibility of correct facts fall to the wayside, they potentially mis-inform all readers.

This means that you, the reader, who wants to utilise systems like ChatGPT, has a massive responsibility on your shoulders. By all means, use the tech! It is amazing, and to keep yourself or your business at the leading edge you want to be able to use whatever advantages you have. However, just like making sure the chemicals used in a process don’t run into the stormwater — it is your obligation to make sure that whatever content you create with ChatGPT (or any other content/decision made with another tool) is correctly informed and reviewed.

So What Can I Do?

The main thing you need to do is review your generated content! Just like reviewing products off a production line for defects, or fact checking your friend with Google, you need to double check!

Take the image from the beginning of the article, I asked ChatGPT to make me a comic on AI Hallucinations. It made a funny comic, but the hallucination was one like you or I may have, not the ones in this article (lets assume that it wasn’t making an artistic statement!). I decided to keep it as I thought it was fitting.

However, if I decided that it was wrong — I would be able to either change the comic myself, or re-generate it by adjusting the prompt with more precise language. This act of tailoring your prompt to produce the ideal output for you is called Prompt Engineering, and is an upcoming field in Data.

However, prompt engineering can only work so much, it can only serve to dampen intrinsic and extrinsic hallucinations, not prevent them. Your best bet is to review everything (especially the statements made as facts!) — not just blindly copy and paste.

Conclusion

Don’t get me wrong, push the envelope. However, before you use the generated content — seriously review what you have. If anything, so that mistakes don’t make you look silly. However, the larger the scale of your work the more responsibility you have to ensure what you’re doing is right.

With modern tech constantly developing, becoming more generally accessible, and exponentially increasing the average persons potential productivity — it is crucial that we do not fall into the what we can do train of thought and focus on the what we should do.

About Me

Hi! I’m Cooper. I am an engineer who moved into data. I believe that the way we implement technology is just as important as the tech itself.

I am thoroughly interested in Computer Vision, MLOps, AgTech and Education.

This year I decided to challenge myself and set about learning to write articles and make videos that communicate science based topics and other miscellaneous topics I am interested in. I hope you enjoy!

I also have a website, check it out!