A Brief Overview of Hallucination in LLM
Large Language Models (LLMs) have shown the impressive capability to understand human language and effectively tackling a wide range of natural language tasks, from text classification to summarization and even answering questions. However, LLMs are observed to have the issue with hallucination, where generating incorrect information or making up the answer which is sometimes difficult for human to notice this behavior.
In this blog post, we would like to help you understand hallucination by explaining why it occurs, how to spot it, and most importantly how to mitigate this issue.
Hallucination in LLM
Hallucination in LLMs is a phenomenon where these models generate incorrect or fabricated information that may not be present in the input data or is not factual. Like other Machine Learning methods, this might be seen as “out-of-distribution” problem.
To understand why hallucination happens, let’s begin with the pretraining phase of LLMs. During this phase, the models are trained on the large amount of data unsupervisedly by predicting the missing word or predicting the next word in the sentence. This process can be mathematically written as follows:
Where y_hat is the next word, y is the possible word, and x is the text input. During this phase, the quality of generating the next word relies on the training data provided. If the training data itself contains incorrect information, it can result in incorrect predictions.
Furthermore, it’s important to note that LLMs are forced to generate the next word in an autoregressive manner. In simpler terms, they have to predict each subsequent word based on their prior prediction.
Moreover, LLMs are intentionally to be stochastic models. This means they’re not limited to just repeating what they’ve seen during training, allowing for more creative outputs.
Detecting hallucination in LLM
Now that we have some background of how hallucination occurs in LLM, let’s explore more on how we can detect these instances. To ensure that LLM-generated content is reliable, here are several ways to identify and address this:
Textual Overlap
Calculating the information overlap between the generated and the reference texts. One of the simple metrics in this group is ROUGE (Recall-Oriented Understudy for Gisting Evaluation) which can be calculated as follows.
Information Extraction Based
By comparing the extracted information (e.g., subject, relation, object) between the generated and the reference texts.
Natural Language Inference (NLI)
Determining whether a “hypothesis” is true (entailment), false (contradiction), or undermined (neutral) given a “premise”.
Question Answering Based
Simply asking the LLM questions related to the generated content and verifying the accuracy of its outputs.
Human Evaluation
Lastly, putting humans to directly evaluate LLMs output.
Detecting the hallucination in LLM is not a straightforward task due to several complexities such as ambiguity in language, lack of ground truth, contextual dependence, etc. Despite all of these, this area is active in research, and many approaches are being explored to tackle this issue. In the following section, we will dive into the high-level framework and explore various strategies for mitigating LLM hallucination.
How to mitigate hallucination
Imagine LLM as a person who doesn’t know everything. Just like humans, LLMs can develop by learning from their mistakes. But how can we facilitate this learning process and help them correct their errors? Thanks to [Pan et al., 2023] for a comprehensive conceptual framework, we can systematically address and mitigate hallucination in LLMs.
Language models, like humans, can encounter issues in their output. These issues might include hallucination, factual inaccuracies, or other undesirable behaviors. To address these issues, we can see it as the process of seeking medical care:
Language Model: Identifying the Symptoms
Think of the LLM’s issues, such as hallucination and incorrect reasoning, as symptoms similar to those of a patient. Just as a patient’s symptoms provide insights into their condition, understanding these issues helps us diagnose the LLM’s problems.
Critic Model: Providing a Diagnosis
The Critic Model can be likened to a doctor or diagnostician. Its role is to assess the LLM’s responses and provide valuable feedback, much like a doctor diagnosing a patient’s condition.
In LLMs context, there are many ways to get feedback — e.g., direct human feedback.
Refine Model: Determining the Treatment
To help or improve a patient’s symptoms, the Refine Model is responsible for determining the best approach to enhance the LLM’s performance based on feedback.
One of the popular techniques is In-Context Learning as it doesn’t require retraining the model, and it can be done in low resource fashion.
Conclusion
We’ve explored the “why” and “how” of scenarios that could result in hallucination, diving into methods for its detection, and discussed a comprehensive framework for mitigation.
Solving hallucination is one of the grand problems in Natural Language Processing (NLP), and there are various aspects to address. We’ve only described the surface. In our next blog post on hallucination, we will dive deeper into specific strategies and research. Stay tuned for a more detailed exploration to enhance the accuracy of these language models.
Acknowledgement
I would like to acknowledge DataX’s AI Research Team for technical and supplementary details. I would also like to acknowledge DataX’s Editorial board for the finetuning process of this article.
