Elevating Conversational AI with Dynamic Data and APIs via OpenAI Function Calling

Unlocking the Power of the RAG Approach in Conversational AI

Many enterprises face the challenge of aligning subject matter experts with projects that match their skillsets and availability, not to mention geographical location and interests. My foremost ambition when I embarked on this technological voyage was to craft a chatbot that could effortlessly streamline project staffing via a natural conversation.

In this article, I’ll walk you through my approach to creating a chatbot that empowers users to query a treasure trove of engineering data sourced from PDF files using regular conversation.

The Foundation: Retrieval Augmentation Generation (RAG)

The project’s architectural foundation is built upon the RAG approach, which significantly enhances the chatbot’s capacity to fetch contextually relevant information and compose precise and informative responses. The integration of function calling, facilitated by the RAG approach, emerged as a standout technical feature, highlighting its game-changing potential in the field of conversational AI.

Step 1: Taming the PDF Beast by Unleashing Azure’s OCR Custom Model

My journey kicked off with the task of wrangling unstructured PDF engineering profiles. To this end, I built a custom Optical Character Recognition (OCR) model trained on relevant engineering PDF profiles. This custom-trained model works its magic by transforming chaotic data into structured, organized information. The outcome? A valuable stockpile of engineering insights primed for the next phase of my venture.

Step 2: Cosmos DB – Efficient Storage

With the structured data in tow, I needed an efficient storage solution. Cosmos DB emerged as a suitable solution for my dataset as it provided seamless integration with Cognitive search. After a profile is processed by the OCR model, it is then ingested into a Cosmos DB container and ready for indexing.

Step 3: Azure Cognitive Search — Precision in Data Retrieval

Azure Cognitive Search offers a comprehensive querying mechanism that allows for the precise extraction of specific data points. This capability empowered my chatbot to navigate the vast sea of information stored in Cosmos DB with exceptional accuracy. The result? User queries are met with responses that are not only accurate but also highly relevant.

Step 4: OpenAI Function Calling — The Pulse of Dynamic Conversations

Function calling was one of the highlights of this project, endowing my chatbot with real-time adaptability. My OpenAI instance adeptly selects which function to use based on the context a user prompt it has received and the description of the function. It then extracts the required structured parameters seemingly from an unstructured user prompt, allowing me to then call my own custom functions with these parameters.

Once the selected function has run, the OpenAI instance is then called for a second time, but now both the original prompt and the function response are used as context. This capability enables the chatbot to generate natural language responses to queries about internal data by grounding custom information at runtime. In my case, this grounded information comes from functions that perform targeted queries against Azure Cognitive Search and external API calls.

A look at three internal functions

In my project, I built the following three internal functions:

1. A function to provide information regarding an engineer’s skill and experience (DB Query).
2. A function to provide information regarding an engineer’s availability (DB Query).
3. A function that retrieves training links from an internal platform (API call).

Conclusion

I’ve merely skimmed the surface of what’s achievable with this project. Weaving together OCR, Cosmos DB, Azure Cognitive Search, and OpenAI function calling with the revolutionary RAG methodology, I’m just beginning to explore the boundless horizons.

This amalgamation lays the foundation for metamorphosing data into a living, breathing AI companion, granting users a frictionless interaction with information. It’s a promising leap towards tackling real-world dilemmas with imaginative and groundbreaking solutions.

Screenshots

Front end Login UI

Chat UI — Showcases a prompt asking about a particular skill (Function 1 from my built internal functions).

Chat UI — Screenshot of prompt asking about a person’s availability (Function 2 from my built internal functions).

Chat UI — Screenshot of prompt asking about training for a particular role in a specific capability (Function 3 from my built internal functions).