How to Test Your RAG Components For Accuracy and Performance.
We built a RAG system in a four-part series a few days back.
We updated and replaced a few components and models based on what worked well for our use case.
But how do we evaluate what we built was right?
For instance, when we write a code we unit test that piece to make sure the piece of code works as we intended. Piece-by-piece when a service is built, we test the service through a tool like postman. Then we try to integrate our service to a client and run integration test.
Once we know the functionality is working as expected, we run security test to make sure only authorized folks can access it, then we run performance test to make sure our service can handle high loads.
Similarly, with a RAG system, we need to find out what to test to ensure our system follows the industry standards and then how to test them.
We can broadly classify the components in a RAG system into two sections
The only component that would be a part of output from LLM is Completion.
Completion is a type of…
