Vector Databases and Vector Libraries (Part 4 of RAG Series)
Storing Embeddings in an efficient manner
This is part 4 of the “Retrieval-Augmented Generation (RAG) — Basics to Advanced Series”. Links to other blogs in the series are at the bottom of this blog. Taking forward from part 1 (RAG Basics), part 2 (Chunking) and part 3 (Embedding) in this blog we will focus on the “Vector Databases and Vector libraries” component in addition we will also include the details around the Retrieval Engine (highlighted in Blue).
In the last few blogs, we have discussed mechanisms to use the text data, break it into chunks and create vectors (Embeddings) of these chunks. Once the (source and query) chunks are converted to vectors, the next step involves comparing the query vectors with the source vectors to identify the closest similarity and extract the relevant chunks from the source. This can be done in two different ways:
In this blog, we will first deepdive into the details and workings of Vector…
