RAG: Graph Retrieval vs Graph Reasoning

Working with our design partners to develop deterministic, explainable and accurate RAG systems, we discover better ways to think about knowledge graphs. Knowledge graphs are not just data stores; they can also be reasoning structures. This is something we have referenced before in our work on Complex RAG techniques with Knowledge Graphs.

Graph Retrieval

Knowledge graphs are conceptually simple things. They are a graph representation of knowledge. This graphical representation can describe the specific facts you want to explicitly represent the connections between. When LLMs consume a document, they must guess the relationships between different entities and concepts mentioned. Because they have to guess these relationships, they sometimes get things wrong due to them being probabilistic. Further, these logical errors compound, as it is not clear if LLMs can reason about uncertainty. However, they become more accurate when the relationships are more defined. Enterprises want more defined structure, so it is easier to ask questions of your data.

For example, in a legal use case, it may be important to note that the definition of the word ‘Defendant’ is explicitly linked to ‘Person A’, to make sure that the LLM never hallucinates an incorrect or spurious answer.

This representation does not need to simply be a concept. This representation may take the form of a document structure, or pages, or any other schema that it makes sense to organize and categorize your information with.

Graph Reasoning

Graphs may also be used as a means to help the LLM understand how to navigate information, instead of simply understanding how information is linked together. These are rules that help enforce, for specific questions, what type of information should be retrieved in what order. ‘Walking the Graph’ or graph walks is another term for this phenomenon.

Creating and enforcing rules for graph traversal is much like a domain-specific workflow endeavor. Information retrieval processes in enterprise settings may be completely arbitrary, depending on the data and workflows being worked on.

For example, we may want to make sure information is being manipulated in a specific decision tree, just like in a Customer Support SOP document, where if certain criteria are met, a specific information retrieval flow is required.

This is a clear distinction from storing all facts within the graph itself, because how the information is retrieved matters almost more than what is in the graph in the first place.

There are some existing academic works on graph-based reasoning that focus on how to translate questions into a graph-based search. At WhyHow.AI, we believe these are useful advancements in the reasoning space, but that many rules for information retrieval processes cannot be reasoned through and have to be manually specified by the domain expert, as well as someone who understands the necessary programming language. An example of this is in representing SOPs, where there may be a specific way that a company wishes to process an information retrieval pipeline.

When do you use Graph Reasoning vs Graph Retrieval?

Graph Reasoning frameworks are utilized when there is a need to deterministically navigate information, inject context, or navigate chunk extraction. Frequently, the use-cases here are “look up” focused, where the answer is known and we simply want to enforce determinism in the way that the LLM system is retrieving answers.

Graph Retrieval is helpful for augmenting additional context, especially around multi-hop use-cases. Frequently, the use-cases here are Search focused, where the answer is not fully known, and we simply want to make sure the LLM is adding additional relevant context for a more holistic answer, on the basis of explicit relationships mapped in the Knowledge Graph.

In both instances, a knowledge graph of your data is required. However, a Graph Reasoning framework may have a more abstract schema, around pages, documents, and SOPs, instead of a graph of concepts, entities, etc. A Graph Retrieval framework is more likely to have a graph of your concepts and entities, since it requires a structured search of your underlying information.

At WhyHow.AI, we are working on tooling for both Graph representation of your unstructured data, as well as deterministic engines for information retrieval through Graph Reasoning engines. If you are looking for deterministic and graph tooling for your RAG pipeline, hit us up at WhyHow.AI or at team@whyhow.ai. Join our discussions about rules, determinism and knowledge graphs in RAG on our newly-created Discord.