Building an LLM tool to better understand scientific papers using RAG

One way to understand a complex scientific paper is to decompose it into parts and explain it to a person that is not familiar with the area of knowledge that the paper belongs to. You might also need to understand other papers that are cited or related to the original paper to add context. As one starts to reading and understand a bit more, one might have questions about concepts or methods that the authors used. Large Language Models (LLMs) are a great tool to tackle this task. In this post, I use llamaindex to use retrieval augmented generation (RAG) to understand scientific papers by being able to ask general questions.

What are RAGs

LLMs are very good at condensing knowledge from all the data they were trained on and present it in a very good way (usually as text). This knowledge can be extracted using clear instructions or questions (prompts). Although these artificial neural networks are pre-trained on a vast amount of texts [1], it would be very useful to add new knowledge to these models so that we can ask questions about particular things that we are interested in, for example novel research papers. One way of doing this would be to fine tune the model, for that it would be needed to retrain the upper layers of the model with the new datasets.

A limitation of this is that training these models is expensive. The reason is that these models are very large (e.g. billions of parameters) so the required hardware (GPUs) need to be…