A RAG system backpacked with Feedback Intelligence

Well, we got to the point… we can take an example of RAG to apply all the theoretical knowledge discussed in the previous article.

TL;DR — LLMs are being used in several ways. We will use the RAG example. In this example, we have a chatbot that is built on top of Gemini with RAG. The chatbot is designed to help students prepare their math skills.

The pipeline is designed to get feedback when a student is not happy with the response. The Connectors capture all kinds of feedback messages ie implicit and explicit and structure them for further analysis (see more here).

Let’s dive deep into one of the feedback messages with the corresponding queries and responses:

Why is the response not correct, what is the issue and root cause?

Let’s use the Insights to analyze it.

The result shows that the root cause of incorrect response is the high probability of a knowledge hole.

Feel free to drop us a line to try it out on your RAG system!

co-author: Haig Douzdjian