Building an AI-Based Product Recommendation Chatbot: A Journey from GPT 3.5/4 models to OpenAI’s Assistants API
TLDR;
Dive into the evolution of AI chatbots with an innovative leap! Prior to the Assistants API by OpenAI, a groundbreaking chatbot was born, blending GPT 3.5 and 4’s capabilities to tailor product recommendations like never before. Imagine a chatbot so smart, it navigates a complex, tree-shaped data model to pinpoint exactly what you’re looking for, whether for personal, home, or business needs. It doesn’t just stop there; with dynamic context generation and a deep understanding of product specifics, this bot crafts conversations that feel remarkably personal. Then came a pivotal shift with OpenAI’s Assistants API, birthing two specialized bots: one that knows the entire product universe, offering broad explorations, and another that zeroes in on specific verticals for lightning-fast recommendations. This journey isn’t just about technological advancement; it’s about redefining the horizon of personalized customer service through AI, opening doors to experiences once thought impossible.
Introduction
In the rapidly evolving landscape of AI technology, our team embarked on a mission to harness the power of GPT 3.5 AND 4 models, predating the release of OpenAI’s Assistants API, to create a sophisticated AI chatbot. This bot was designed to offer highly personalized product recommendations, navigating through a complex web of user needs and expectations.
The Genesis of Our AI Chatbot
Our journey began with the ambition to maximize the 32K context window limitation of GPT 3.5. We devised a dynamic context generation strategy, rooted in a meticulously structured tree-shaped data model. This model comprises nodes, each holding vital information including metadata, summaries, use cases, and, for leaf nodes, specific properties of products or services.
The Data Structure
The foundation of our chatbot’s intelligence lies in this hierarchical arrangement:
- Root Node: Represents out full catalogue.
- Level 2 (branch): Categorizes three primary user intentions — personal, home, and business needs.
- Level 3 (branch): Delve into specific verticals like dating apps, home security, or business services.
- Level 4 (leaf): Detail individual products or services, enriched with unique properties and use cases.
This structure enables the bot to guide users through the intent discovery part of the chat, with precision, tailoring recommendations to their exact requirements.
Dynamic Context Generation
Utilizing a function named `get_node`, the bot dynamically crafts the conversation’s context by selecting the appropriate node based on user responses.
{
"name": "get_node",
"description": "Get the child data node required for continuing the conversation. Do not use this function for children that are of type leaf. Only use this function to get branch nodes.",
"parameters": {
"type": "object",
"properties": {
"namespace": {
"type": "string",
"description": "the namespace of the next node."
},
"path": {
"type": "string",
"description": "The path of the child node as provided on the children array in the 'path' field. Use the exact same value as provided on the relevant child node 'path' field. For example: '/node-path'."
},
"summary": {
"type": "string",
"description": "The detailed summary of the current conversation history"
}
},
"required": [
"namespace",
"path",
"summary"
]
}
}Creating the Data Structure: A Detailed Look at Our Process
In developing our AI chatbot, the creation of the underlying data structure plays a pivotal role. This structure, akin to a multi-tiered hierarchy, consists of leaf nodes (products) at its most granular level, ascending through verticals and categories up to the “Top10” root node. Here, we delve into the meticulous process of constructing this hierarchy, focusing on the generation of leaf nodes and the aggregation of data as we move up the structure.
The Leaf Level: Product Nodes
At the heart of our data structure lies the product node, representing the leaf level. Each product node is a rich repository of information, comprising:
- Metadata: Essential identifiers like the product’s name, ID, and other unique characteristics.
- Summary: A concise description capturing the essence of the product.
- Use Cases: A list of scenarios or applications that differentiate the product from its counterparts.
- Key Properties: Critical attributes that define the product’s utility and appeal.
Creating a Product Node with GPT 4: To populate these product nodes, we leverage the capabilities of GPT 4. By crafting detailed prompts that outline the task and incorporating raw data from our product review pages, we enable GPT 4 to generate comprehensive and coherent product nodes.
product_node_messages = [
{"role": "system",
"content": "You are an assistant of a chatbot that assists online "
"users by recommending the right product/service for their "
"needs and expectations."},
{"role": "system",
"content": f"The product summary is:\n{product_review_content}"},
{"role": "user",
"content": "Return as many properties and use cases as possible with "
"a maximum of 10 for each."}
]*The above prompt is a simplified version of the original prompt. the original prompt is much more detailed.
As we are working in a server-to-server mode, we also want to get the data in a predefined structure. For this purpose, we always use a function to deliver the result. You can read about functions here.
Here’s the function schema used for the product “leaf” node:
product_node_functions = [
{
"name": "create_product_node",
"description": "Create a new product node with the name, id, properties and use cases as generated by the chatbot assistant.",
"parameters": {
"type": "object",
"properties": {
"name": {"type": "string", "description": "the name of the product."},
"id": {"type": "string", "description": "the id of the product."},
"summary": {"type": "string", "description": "A short summary of the product."},
"properties": {
"type": "array",
"items": {
"type": "string"
},
"description": "A list of product properties that differentiate this product."},
"use_cases": {
"type": "array",
"items": {
"type": "string"
},
"description": "A list of possible user use cases that represent users that may want the use the "
"product."
}
},
"required": ["name", "id", "properties", "use_cases", "summary"]
}
}
]*We pass an array of functions as Openai’s api accompanies multiple function schemas in a single call.
Aggregating Into Verticals
Once individual products within a vertical are defined, the next step involves synthesizing this information to create a coherent summary for the vertical itself. This includes a distilled list of use cases highlighting the common or complementary functionalities across the product range.
Creating Vertical Summaries with GPT 4: Using GPT 4, we input the collated list of product nodes to generate a succinct summary of the vertical, alongside a “combined” and refined list of use cases. This step ensures that each vertical is represented by a narrative that reflects the collective strengths and unique propositions of its constituent products.
node_summary_messages = [
{"role": "user",
"content": f"Create a summary of the node\n "
f"Node name: ```\n{node_data.get('name', '')}\n```\n"
f"Node content: ```\n{node.get('summary', '')}\n```\n"
f"Node children: ```\n{node.get('children', [])}\n```\n"},
{"role": "user",
"content": "In your summary, combine the list of the children's use cases "
"to a single list that aggregates all of them without repeating any of "
"them."},
{"role": "user",
"content": "In your summary, the use cases should describe the business "
"need that the node covers.\n"
"For example: Adding a crm system to your website."}
]As with the product node, we also use a function to get a json formated output here:
node_summary_functions = [
{
"name": "create_node_summary",
"description": "Create a new node summary by combining child nodes use "
"cases as generated by the chatbot assistant.",
"parameters": {
"type": "object",
"properties": {
"name": {"type": "string", "description": "the node name."},
"summary": {"type": "string",
"description": "A short summary describing the node's content."},
"use_cases": {
"type": "array",
"items": {
"type": "string",
"description": "A child node use case description."
},
"description": "A list of possible user use cases aggregated from the children nodes. Use "
"cases should be broad so that each of them combines a few closely related "
"child node use cases."}
},
"required": ["name", "summary", "use_cases"]
}
}
]Building the Hierarchy
The process repeats as we ascend the hierarchy, with each level aggregating and synthesizing information from the level below. From verticals to categories and finally to the top node, each step involves summarizing and distilling information to ensure a clear, comprehensive representation of the entire product landscape.
Creating Higher-Level Summaries with GPT 4: At each higher level, the aim is to capture the overarching themes, use cases, and attributes that bind the lower levels together, thereby creating a unified and navigable structure. GPT 4’s generative capabilities facilitate this synthesis, ensuring that our chatbot’s data structure is both deep and wide, encompassing a broad spectrum of products and services while maintaining detailed insight into each.
namespace_summary_messages = [
{"role": "user",
"content": f"Create a combined summary of the namespace: \"{namespace}\"
f"from the following verticals nodes:```\n{namespace_children}```\n"
f"In your response, provide a detailed aggregated list of "
f"use cases that match the use cases provided on each node.\n"
f"Include all child verticals in your summary."}
]This structured approach to building the data hierarchy not only enhances the chatbot’s ability to provide tailored recommendations but also ensures that the bot remains adaptable and scalable. By systematically organizing and summarizing data, we lay a solid foundation for our AI to understand and navigate an ever-expanding universe of products and services, thereby delivering personalized and relevant recommendations to our users.
We use the same “node_summary_functions” as for the vertical nodes for defining the output structure.
Transition to OpenAI’s Assistants API
The release of OpenAI’s Assistants API marked a pivotal shift in our approach. We adapted our infrastructure to leverage this new API, resulting in two distinct types of bots:
1. Full Tree Bot: Utilizes the API’s file retrieval capability to navigate the comprehensive product tree stored in periodically updated files.
2. Vertical-Specific Bots: Crafted for specific user contexts, like seeking meal delivery services, providing focused recommendations based on the user’s location.
At this point, there was no longer a need for the “get_node” function as the full Tree Bot internally uses RAG built from the knowledge files we upload to it, that hold our full tree.
Tailoring Experiences: The Dual-Bot System
Our approach in deploying two distinct types of AI chatbots — the Full Bot and the Single Vertical Bot — showcases our innovative strategy to cater to diverse user needs with precision and efficiency.
The Full Bot is designed to guide users from a broad starting point, adeptly navigating through our extensive product tree to pinpoint the perfect product or service match. This comprehensive exploration, however, necessitates querying the Retrieval-Augmented Generation (RAG) model, which introduces latency as a trade-off for its depth and breadth of analysis. It’s an ideal solution for users whose specific needs or preferences are not initially clear, providing a thorough discovery journey tailored from the ground up.
Conversely, the Single Vertical Bot offers a streamlined and rapid response, catering to users with a defined intent. By focusing on a specific vertical, this bot sidesteps the need for extensive data retrieval, significantly reducing latency and directly enhancing the user experience. This efficiency makes it exceptionally suited for scenarios where the user’s interest area is already known, allowing for a quick and personalized product recommendation process.
This dual-bot framework underscores our dedication to versatility and user-centric innovation, ensuring that regardless of the user’s starting point or intent, our system delivers tailored, effective, and timely recommendations.
So which approach is better?
This question should be addressed by comparing the key KPI’s of our bots. Let’s assume that both types of bots get users from A to Z in the same flow, which is a very bold assumption but let’s consider this as true, so it all comes down to speed to cost ratio. We want the bots to respond fast and we want them to be affordable.
Theoretically, and as we initially expected, you’d think that a short instruction text combined with a good RAG would be both fast and efficient. However, as we discovered, the RAG process is both time and token consuming. I’m sure that there’s much to be optimized and personally I believe that this would be the way to go, but our current tests show us that the cost is about the same but the response time “penalty” of using RAG is still high. That’s why we’ve created the “specializing” bots which carry the entire list of products in their instructions.
Conclusion
The development of our AI chatbot has been a journey of technological evolution, from the initial use of GPT 3.5 to the integration of OpenAI’s Assistants API. Through innovative data structuring and adaptive use of technology, we’ve created a solution that not only meets the diverse needs of users but also paves the way for future advancements in AI-driven customer service.
As we continue to explore and expand the capabilities of our AI chatbot, the potential for creating more sophisticated, user-centric platforms becomes increasingly evident. Our journey underscores the transformative power of AI in reshaping how businesses interact with their customers, offering personalized experiences that were once beyond reach.
