Basics Of RAG (Retrieval Augmented Generation) — GENERATION

1. Adding Docs to Context Window

Adding Docs to Context Window involves incorporating relevant retrieved documents into the context that an AI model uses to generate responses:

• Context Window: This is the segment of text that the model considers when generating a response. In the case of large language models (LLMs), this window is limited in size, so only the most relevant information is included.
• Incorporation: After retrieving the relevant documents, they are added to the context window alongside the original query or prompt. This enriched context helps the model generate more accurate and informative responses.

Think of it like providing an AI with a relevant chapter from a book along with a question, so it has more detailed information to give a better answer.

Adding docs to context window

2. Connecting Retrieval with LLMs via Prompt

Connecting Retrieval with LLMs via Prompt means integrating the retrieval process with the language model by feeding the retrieved documents into the model through a prompt:

• Retrieval: First, the system retrieves the most relevant documents based on the similarity search.
• Prompt Construction: These retrieved documents are then combined with the user’s query to form a comprehensive prompt that is fed into the LLM.
• Response Generation: The LLM uses this detailed prompt, which includes both the query and the additional context from the retrieved documents, to generate a more informed and accurate response.

Imagine you’re asking a librarian a question, and they provide you with a specific book or article along with their answer to give you a more complete and accurate response.

Connecting Retrieval with LLMs via Prompt