Dive Deep into RCB Matches with RAG
Note: I have a penchant for grasping new ideas through analogies and enjoy passing on that understanding in the same manner.
That’s why, in this blog post, I’m going to introduce the concept of RAG through an engaging analogy, followed by a practical guide on constructing a RAG for Q&A over RCB matches in IPL 2024.
What is a RAG?
Retrieval-Augmented Generation (RAG) is an approach similar to what we human beings use for answering questions. When someone asks me “What is a cellphone?”. My brain finds all the information related to cellphone in my memory and then it simply uses these pieces of information to build the answer ”It is device that is essentially used for communication, but recently it can also be used for multiple uses- social media, listening to songs, recording audio etc”. This is exactly what a RAG system does, it will use the query to find relevant documents and then these only these relevant documents are passed to a LLM to compose the answer.
Why RAG, if I can ask questions directly to large language models (LLMs)?
Well, imagine you have a lengthy PDF report with hundreds of pages and you want to ask questions about it. If you try to use an LLM directly, you’d have to give it the entire PDF as context and then ask your questions. However, most LLMs can’t handle context longer than 32,000 tokens. This is where RAG comes in. It can take your question, find the relevant parts of the PDF, and use only that information to generate an answer. This saves the cost of using an LLM for the entire document and makes it more efficient.
So lets get started….and understand it in five steps as shown in the Figure 2
Step 1: User enter the prompt + query
eg: “ What is a cellphone, please explain?” (in this part the query is the “cellphone”)
Step 2: The query “cellphone” is passed to our knowledge base
Step 3: The knowledge base returns the relevant information regarding cellphone, lets call it “enhanced context”
Step 4: This is where we send the user prompt + query + enhanced context to the LLM and ask it to generate the response to the query given enhanced context
Step 5: The LLM uses the enhanced context and composes the response in a way that is comprehensive to the user.
Now, lets collect the data on RCB matches and then start build a RAG…
Data on RCB Matches
To get full information on RCB matches, I decided to download the full commentary on cricbuzz into a txt file
So I have two data files as RCB played 2 matches till now (27 March 2024). My data files are RCBvsCSK.txt and RCBvsPBKS.txt
Lets start building a RAG system using llama-index framework….
- Install llama-index python package
2. Import the packages/libraries and enter your OpenAI API Key
3. Load the documents
Your data could exist in various formats such as txt, PDF, Wikipedia, HTML, or others, but for llama-index, it needs to be stored in the document object type. That’s why llama-index provides several document readers capable of extracting text from PDFs, txt files, and other formats, and then loading it as a document object. One such reader is the SimpleDirectoryReader that is used for reading PDFs, text files and many other formats.
Lets check if the document is loaded correctly
4. Creating Index and Retrieval
This a very interesting step and lets use analogy here again to understand this step.
When our brain learns about “cellphone” for the first time, it tries to understand it by various means, like touching it, listening music, checking apps, sending messages etc. This entire process carried out by brain is called indexing, where the information on “cellphone” is stripped into [touch, feel, sound, shape] features and stored in the memory.
Now lets understand retrieval, close your eyes and say “cellphone” (try it now). Did you see an image of a cellphone- that is retrieval, your brain sent the query “cellphone” and retrieved an image.
Similarly, in RAG, we need to create index of the documents so that we can retrieve only the relevant information from them when we query. Also, we are aware that the computers stores information in numerical format so that during retrieval we can find relevance score and return relevant text, which is why we are going to use embedding model to convert the document text into numbers and then create the index.
5. Query Engine
In this step, we are going to build a query engine on top of the index that we created, which means that when we pass a query to this query engine, it will retrieve the relevant docs and then these relevant docs along with the prompt and query will be sent to the LLM to compose a proper response
That’s it, let the Q&A begin! 🚀🔥
Isn’t this amazing? With just a few lines of code, you can query your own documents and get almost perfect responses! 🤯
Now, let’s talk about complex questions. For example, if you want to ask a question like: “Compare Virat’s performance from both matches and also provide a list of bowlers with the lowest economy?”
At such times, the RAG we built might struggle because it needs to gather relevant text from both matches for Virat and then collect bowler stats. This is where the “SubQuestionQueryEngine” comes in! 💡
Sub Question Query Engine
It breaks your query into multiple questions and then RAG fetches answers to all these sub question to compose the final response
While using the SubquestionQueryEngine, we perform a slight modification to the code. We build a query engine tool that allows RAG to query from 2 separate docs.
Lets ask complex question based on two matches…
As seen above, there are 4 sub questions generated from our query and using the answers to these queries, the LLM builds the final answer.
Conclusion:
The aim of this blog was to understand RAG by means of analogy and also build a simplistic RAG with llama-index.
Hope this blog gets you started with building your first RAG! 🚀
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Connect with me on Linkedin: https://www.linkedin.com/in/mahantesh-pattadkal-7a2607101/