Understanding Retrieval Pitfalls: Challenges Faced by Retrieval Augmented Generation (RAG) models

Improving the performance and application of Large Language Models

Image generated with Google’s Gemini, 24 February 2024.

Author

• Amanda Kau (ORCID: 0009–0004–4949–9284)

Introduction

Large language models (LLMs) like GPT-4, the engine of products like ChatGPT, have taken centre stage in recent years due to their astonishing capabilities. Yet, they are far from perfect. Many of us have since learnt — perhaps when asking ChatGPT a question or employing it to write our reports — that LLMs can hallucinate. This happens when the LLM so eloquently expresses false knowledge that we might be fooled by it. This major flaw has spurred the popularity of Retrieval-Augmented Generation (RAG) techniques as a way to optimise an LLM’s responses.

To start off, this article will cover a brief overview of the key concept behind RAG. Subsequently, a review on several issues behind the retrieval step of RAG will be presented. In particular, this article will review ideas on when and what should be retrieved, the quantity of retrieved documents, effects of data quality, and RAG applied to different domains. Strategies proposed by the research community will also be briefly introduced for each…