RAG 2.0: Retrieval Augmented Language Models

Language models have led to amazing progress, but they also have important shortcomings. One solution for many of these shortcomings is retrieval augmentation. There have been plenty of articles written about Retrieval Augmented Generation (RAG) pipelines, which as a technology is quite cool. But today, we are taking it one step further and truly exploring what’s next for the technology of RAG. What if we can create models with trainable retrievers, or in short, the entire RAG pipeline is customizable like fine-tuning an LLM? The problem with current RAGs is that they are not fully in tune within it’s submodules, it’s like a Frankenstein monster, it somehow works, but the parts are not in harmony and perform quite suboptimally together. So, to tackle all the issues with Frankenstein RAG, let’s take a deep dive into RAG 2.0.

Table of Contents

• What Are RAGs?
• What RAG 2.0 Achieves?
• How Does RAG Solve Issues of Intelligence?
• Better Retrieval Strategies
• SOTA Retrieval Algorithms
• Contextualizing the Retriever for the Generator
• Combined Contextualized Retriever and Generator
• SOTA Contextualizaton
• Conclusion

Please take a moment to share the article and follow me, only if you see the hard work. It takes weeks to write such articles, some love would be highly appreciated.