Increase the Value of your Knowledge Base with AI
All companies that grow beyond the first few employees and customers reach a point where governance and routines become as important as the products they sell or services they provide. What then follow is documentation and instructions for how processes work and what routines to follow.
There is nothing wrong with this, but there are some practical challenges and inefficiencies related to it. One challenge might be just finding the right document, or there might be conflicting instructions related to the same routine.
In this article, I will review the technical aspects of what it takes to combine your knowledge base with generative AI to create tailored and specific answers for your situation. A process known as Retrieval-Augmented Generation or RAG for short.
Disclaimer: do not do this without considering the security and compliance implications of sharing your data with a third party service provider.
Technical Toolkit
There are three distinct components that need to cooperate to create a service that supports RAG. A vector store, an AI implementation and your service that ties the other two together.
In this example I will use PostgreSQL with the pgvector extension and OpenAI and Spring AI as the service framework. Below are the dependencies required to run this example:
<!-- Spring AI -->
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-pdf-document-reader</artifactId>
</dependency>
<!-- Vector store -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-pgvector-store</artifactId>
</dependency>Setup your AI service
Our first step is to select an AI service provider and get the necessary credentials to call their API:s. Here we will use OpenAI as our service provider and we start by registering to use their API:s which will also require a credit card.
Once everything is setup you register a new API key and in a terminal run
export SPRING_AI_OPENAI_API_KEY=<key>Data and Vector Storage
We will use an Official Report of the Swedish Government, specifically SOU 2023:41 which is an investigation about collective agreements and unemployment insurance.
The first step will be to make the information in the document available to our service, more specifically to store the data in such a way that we can find the relevant pieces of the document when a user asks our service a question.
To do this, we create a VectorStore in our service that will be used to store our document content.
@Configuration
public class VectorStoreConfig {
@Bean
public VectorStore vectorStore(JdbcTemplate jdbcTemplate, EmbeddingClient embeddingClient) {
return new PgVectorStore(jdbcTemplate, embeddingClient);
}
}The EmbeddingClient is used to give each Document that we want to store in a vector representation. This operation is provided by OpenAI so for each Document that we ingest to our store, there is a small fee for calculating the corresponding vector representation.
// Download file from https://www.regeringen.se/contentassets/533e95b43d68406b86ed42d9188f3080/sou_2023_41_pdf-a_webb.pdf
var pdfReader = new PagePdfDocumentReader(
"file:src/main/resources/sou_2023_41_pdf-a_webb.pdf",
PdfDocumentReaderConfig.builder()
.withPageTopMargin(0)
.withPageBottomMargin(0)
.withPageExtractedTextFormatter(ExtractedTextFormatter.builder()
.withNumberOfTopTextLinesToDelete(0)
.withNumberOfBottomTextLinesToDelete(0)
.withNumberOfTopPagesToSkipBeforeDelete(0)
.build())
.withPagesPerDocument(1)
.build());
var documents = pdfReader.get();
log.info("Got {} documents", documents.size());
vectorStore.add(documents);Running this code against the linked PDF document, which contains 290 pages, takes more than two minutes. It is the process of giving each page a vector representation, i.e. calling OpenAI, that takes most of this time. Once completed, your database will be populated with an entry for each page together with the text and a point in space representing the text on that page.
[0.0025787537,-0.013284697,0.00701956,-0.022029143,-0.02063003...]Processing User Questions
Now that we have populated our vector store with information, we are ready to process user input.
var search = SearchRequest.query(question);
var searchResult = vectorStore.similaritySearch(search);
log.info("Search done, found {} documents", searchResult.size());When invoked, the question asked by your user is first used to find relevant documents that match their question. This is done by first using the same algorithm that was used above, i.e. invoking the OpenAI API and then finding the nearest documents in the vector store.
Creating a Prompt
Our next step is to create a Prompt that we can use in our interaction with our LLM. This is where prompt engineering is required and in our use case, we gave it a simple instruction
var promptTemplate = """
You work as an investigator at the Swedish parlamentary office.
Your job is to answer questions from the public and journalists.
Use the information from the DOCUMENT-section below to give
complete answers.
DOCUMENTS:
{documents}
""";
var joinedMessages = searchResult.stream()
.map(Document::getContent)
.collect(Collectors.joining("\n"));
var promptTemplate = new SystemPromptTemplate(ourPromptTemplate);
var systemMessages = promptTemplate.createMessage(Map.of("documents", joinedMessages));
var userMessage = new UserMessage(question);
var prompt = new Prompt(List.of(systemMessages, userMessage));
var chatResponse = chatClient.call(prompt);In the code, we join the information we found in our vector store based on the user question, the instruction to our AI and the question posed by the user. This is then sent to OpenAI and a response returned.
A question about which persons had the role of secretary in the public investigation 2023:41, yields the following result
> Which persons had the assignment of secretary in SOU 2023:41?
In SOU 2023:41, the following persons had an assignment as secretary:
- F. Karlman was appointed as secretary from the 1st of January 2022.
- L. Ahlström was appointed as secretary and unit manager
J. Hedelin as main secretary from the 1st of April 2022.
- L. Ahlström was dismissed on the 13th of December 2022.
- Investigator C. Vestin was appointed as secretary from the 13th of
February 2023.
- Department secretary K. Lindström was appointed as secretary from the
16th of May 2023 until 30th of June 2023.
> Which persons had the assignment of secretary in SOU 2023:41?
In SOU 2023:41, the following persons had an assignment as secretary:
- F. Karlman was appointed as secretary from the 1st of January 2022.
- L. Ahlström was appointed as secretary and unit manager
from the 1st of April 2022. She was dismissed on the 13th
of December 2022.
- C. Vestin was appointed as secretary from the 13th of
February 2023
- K. Lindström was appointed as secretary from the 16th of May 2023
until the 30th of June 2023.As you can see, the same question asked a few seconds apart gives slightly different answers, most notably it gives different interpretations of the roles that C. Vestin and K. Lindström had. At this stage, we have not tuned our chat client at all or put any effort into our prompt, which will follow in an up coming article.
