Unlock your enterprise data with Retrieval Augmented Generation (RAG) using watsonx Assistant, Watson Discovery & watsonx.ai

The Need

Consider the following scenario

Employee : wants to know the vacation dates for the current year in their location and in their client location.

Insurance customer : wants to know the documents they need to attach during their claims submission.

Credit card customer : wants to know the benefits of the credit card and if lounge facility in an airport is available for them ?

Solution

The answers to above questions are present in enterprise documents. To provide relevant, natural language answer, it is important to search for most relevant answer from up to date enterprise documents and present it in natural language. This is achieved by a pattern called “Retrieval Augmented Generation”.

One way of implementing this is by using watsonx Assistant, watson Discovery and watsonx.ai as shown below.

RAG using IBM watsonx Assistant, Watson Discovery and watsonx.ai (image created by author)

In the above solution architecture,

watsonx Assistant : is used for building AI-powered virtual agents without coding. The conversations once developed are available across multiple channels such as IVR, web, email and others. It integrates seamlessly with multiple enterprise systems to leverage both structured and unstructured data present in the enterprise in answering customer questions. It integrates with Discovery and watsonx.ai

Watson Discovery : All enterprises, be it a large bank, telecom, retail has huge amount of unstructured data present in their enterprise systems such as SAP, on their web portals. Examples of unstructured data include operational documents, Product details, Standard operating procedures, reports. These documents can be leveraged to answer customer questions. however they need to stay within enterprise. In this solution, Watson Discovery provides the ability to understanding the document structure, searching these documents using natural language. Watson Discovery returns the top results (passages containing answers) for each search query along with title of the document and the passage from which answers were taken. This is very important and essential in order to establish trust and transparency with end user by showing them where the answer came from.

watsonx.ai : Multiple foundation models are made available by companies and open source community every day. This will continue to increase. In the above solution, watsonx.ai is used to experiment with different foundation models available from open source and select the one that provides required accuracy and cost efficiency for this scenario where the foundation model need to take the question and top answers, re-rank them and provide natural language answers to the end user. Selecting the right model and prompt engineering is an iterative approach. watsonx.ai enables AI engineers to do these iterations with ease in the prompt lab.

Considerations when leveraging RAG in enterprises

• Leveraging enterprise data can help in answering customer questions quickly. Watson discovery is a good starting point in the above solution. Start with one set of documents ( ex: product manuals ). Use these documents with Watson Discovery, try semantic search and test answers to ensure it will satisfy customer queries. working closely with customer care business unit in understanding the type of questions coming from customers can help in identifying low hanging fruits.
• There are many foundation models out there now and more will be coming, Each of the foundation models are generic and can support multiple tasks such as summarization, classification, Q&A, text generation among others. Select the model which supports Q&A to start with. High parameter model is a good choice to start with.
• Curate the test questions and answers. These can be real questions from customers or the one that represents different types of questions.
• Prompt engineer the model to get the results close to your test data. This can take some time. But having good understanding of prompt engineering techniques can help here.
• Select lower cost models and same prompts to see if they give similar answer.

Implementing RAG using watsonx Assistant, Watson Discovery & watsonx.ai

Enterprises and Engineers and learning enthusiasts can try implementing RAG on IBM Cloud using free/trial version of watsonx Assistant, Watson Discovery & watsonx.ai. Stay tuned to next article providing details of the implementation which you can use to get hands-on experience.