When Machines Imagine: The Phenomenon of AI Hallucinations
There is a large problem that haunts the machine learning landscape, one that calls into question every single bit of content generated by artificial intelligence. In a landscape where misinformation is a looming threat, AI hallucinations emerge as stealthy culprits, raising questions about the trustworthiness of every output crafted by artificial intelligence.
We’ve all been there before, working away at a problem that we seemingly have no solution for, or learning about a topic that doesn’t have as much information available as we would like. Preparing for a crucial exam, you might turn to your AI-powered study partner for a quick quiz review, only to receive answers that seem like they’ve been plucked from a parallel universe. Concepts you’ve never encountered, dates that don’t align, it’s as if the AI is testing you on a different subject entirely.
This peculiar phenomenon, known as a “hallucination”, highlights intriguing complexities at the intersection of technology and education. Understanding and addressing these quirks will become pivotal as we increasingly rely on them in the future.
Basic breakdown
So what exactly IS an AI hallucination? In the most simple terms, it is a response generated by an AI which contains inaccurate information of some kind. These hallucinations can seem very illogical and unexpected in nature, and it can be hard to track exactly what causes them.
We can most commonly see these errors through LLMs such as OpenAI’s ChatGPT and Google’s Bard, as users interact with these products in advanced conversation millions of times a day. If you’ve used these services before, you’ve probably run into these hallucinations before, whether you realized it or not.
What are the dangers of hallucinations?
But some of you may be inclined to ask, “What’s the big deal? Don’t they warn us about this stuff anyway?” You would be right!
In OpenAI’s usage policies, they explicitly state that generated content through their platform should not be used for many professional purposes, including financial, medical or legal advice.
“OpenAI’s models are not fine-tuned to provide legal advice. You should not rely on our models as a sole source of legal advice.”
However, not everyone has taken heed of these disclaimers, and some have faced dire consequences. One example of AI hallucinations backfiring can be seen in the case of two New York lawyers submitting a legal brief that included six fictitious case citations generated by an artificial intelligence chatbot, ChatGPT. In short, the lawyers blindly believed the citations falsely generated by the chatbot and failed to properly verify their sources.
Situations like this do not arise simply from the use of AI, but from overdependence on the technology. This does beg the question; What practices need to be enforced in order for AI to be used as a reliable tool?
Dealing with hallucinations
The most straightforward and guaranteed way to combat AI hallucinations is through the use of manual fact checking.
For example, programming with an AI assistant can be made foolproof through consistently checking documentation. Using this on a large scale for business purposes, however, can be slow and time consuming.
Because of these issues, reputable organisations are starting to gear their efforts towards keeping hallucinations in check. One prominent software being used now is Nvidia’s NeMo Guardrails, an open source project designed to make sure that the output of large language models is more accurate than ever before. Nvidia’s noble mission statement is to keep “smart services aligned with safety, privacy, and security requirements so these engines of innovation stay on track”
Minimizing Hallucinations
We now have a better idea of what hallucinations are, how they can negatively impact our work, and how to get around them. However, a responsibility falls on the developers of AI models to minimize the amount of hallucinations generated in the first place.
Factors that can cause AI hallucinations include insufficient and outdated training data, new untested information and even things as deliberate as “attack prompts” meant to confuse an AI with deliberately harmful prompts.
Lucky for us, there are ways to combat these factors. According to experts at Zapier, we can narrow our prompt answers down, focus on relevant data and make sure prompts are specifically engineered to discourage our models from lying to us. When designing a model, it is also important to limit how “creative” our model is allowed to get. There are other methods that go more into depth on the matter here.
New techniques are being progressed every day as organizations strive to improve their models. OpenAI has recently adopted a new method of dealing with hallucinations known as “process supervision”, a complex process that rewards proper reasoning in every step of an output. In an interview with CNBC, OpenAI researcher Karl Cobbe states that “The motivation behind this research is to address hallucinations in order to make models more capable at solving challenging reasoning problems.”
Conclusion
Hallucinations are a concern with every response generated by AI, but with work being done on both ends of the process, the impact of these errors can be greatly mitigated. Until there is a foolproof way of verifying everything, there must be an asterisk next to all AI generated content. As users, it is our responsibility to make sure we vet the information we use without recycling inaccurate results and contributing to misinformation. Likewise, as designers, it is our job to train our models responsibly and to consistently improve on inaccuracies as soon as we can discover the cracks within our models.
This article was written for QMIND — Canada’s largest undergraduate community for leaders in disruptive technology.
