Building your own OpenAI-powered Conversational ChatBot
Lately, there has been a growing interest in chat-based assistants capable of solving various problems, such as assisting end customers with self-help features or aiding project teams. When developing a chatbot, there are three crucial factors to consider:
- Data Input: It’s essential to provide the chatbot with as much relevant data as possible to enable it to assist users effectively.
- Data Searchability: The next step involves ensuring that the data is easily searchable and that the results presented to users are pertinent to their queries. To achieve this, we will employ a Vector DB for data searchability.
- Meaningful Results: Ultimately, the results presented to end users should be in a user-friendly and understandable format.
In this post, I will guide you through the process of creating your own chatbot using OpenAI (for generating embeddings and LLM support) and the Milvus database as a Vector Database.
Architecture
Before getting into the actual implementation, let us take a look at the high-level architecture
It’s crucial to grasp the significance of vectorization in this context. The fundamental principle at play is the transformation of text into an N-dimensional vector, which captures not just the literal meaning but also the contextual associations within the data. The advantage of comparing vectors lies in its speed and the flexibility to perform comparisons using various criteria, such as cosine similarity or Euclidean similarity-based searches.
Pre-requisite
To effectively store and retrieve vectorized data, a Vector Database is essential. In this instance, I’m utilizing Milvus as an example. Milvus database excels at both storing and comparing vectors, forming the core of the search feature that ensures relevant information is delivered to the end user.
When it comes to transforming textual data into vectors, there are various techniques at our disposal. For this particular case, we’ll harness the Embeddings service offered by OpenAI.
Data Preparation
The first step is to feed the data into the Milvus database. This itself means reading records and converting the textual data into vector format. We store Vectors along with original data in the Milvus database.
Create Collection
FieldType fieldType1 = FieldType.newBuilder()
.withName("employee_id")
.withDataType(DataType.Int64)
.withPrimaryKey(true)
.withAutoID(false)
.build();
FieldType fieldType2 = FieldType.newBuilder()
.withName("employee_data")
.withDataType(DataType.VarChar)
.withMaxLength(2000)
.build();
FieldType fieldType3 = FieldType.newBuilder()
.withName("employee_vector")
.withDataType(DataType.FloatVector)
.withDimension(1536)
.build();
CreateCollectionParam createCollectionReq = CreateCollectionParam.newBuilder()
.withCollectionName(collectionName)
.withDescription("Employee search")
.withShardsNum(2)
.addFieldType(fieldType1)
.addFieldType(fieldType2)
.addFieldType(fieldType3)
.build();Insert Data
List<InsertParam.Field> fields = new ArrayList<>();
fields.add(new InsertParam.Field("employee_id", app_id_array));
fields.add(new InsertParam.Field("employee_data", linearr));
fields.add(new InsertParam.Field("employee_vector", search_vectors));
InsertParam insertParam = InsertParam.newBuilder()
.withCollectionName(collectionName)
.withFields(fields)
.build();
milvusClient.insert(insertParam);Search
As we have stored the data in vector form in our database we first want to convert the query itself to vector. It is important to use the same approach to create the vector for the query as was used for the original data, otherwise, the search will not work. Once we have a vectorized query, we query the database for the nearest top K results. We can choose the value of K based on requirement.
List<String> search_output_fields = Arrays.asList("employee_data");//lines
List<Float> vectors = llmUtility.getEmbeddings(query);
List<List<Float>> search_vectors = Arrays.asList(vectors);
SearchParam searchParam = SearchParam.newBuilder()
.withCollectionName(collectionName)
.withMetricType(MetricType.L2)
.withOutFields(search_output_fields)
.withTopK(3)
.withVectors(search_vectors)
.withVectorFieldName("employee_vector")
.build();Preparing the Results
Once we have the results back, we might want to enhance the response for better human readability. We can use OpenAI’s chat completion API for this purpose.
line = "Please generate text summarization for following data -- " + resulttext;
HttpResponse<JsonNode> response = Unirest.post(chatEndpoint)
.header("Content-Type", "application/json")
.header("Authorization", "Bearer "+key)
.body("{\n \"model\": \"gpt-3.5-turbo\", " +
"\"messages\":[{\"role\": \"user\",\"content\": \"" + line + "\"" +
"}]}")
.asJson();Complete Code Example
Enough of theory let's dig into the code
ChatBotController: Provides functionality for inserting data into the Milvus database, validating data (printData), and searching the Milvus database based on a given query string.
LLMUtility: Helps to connect to OpenAI endpoints and provides embeddings and chat-based summarization. One needs to provide a valid OpenAI key.
MilvusUtility: Implements Milvus-related functionality to insert, validate, and search the data. The sample code uses an employee.csv file to insert records into the database. Make sure to provide the correct host, port, and token/ authorization (if connecting to the hosted version).
Enhancements possible.
It’s important to note that, in this scenario, we’re not utilizing past search information, and every search is treated as a fresh query. However, it’s possible to enhance the system by introducing search history management, enabling more contextual searches based on prior queries. For instance, if the initial query is “Give me details of the XYZ project?” and the subsequent one is “Who all works on this project?” we can accomplish this by maintaining a record of historical search queries and their corresponding responses, which can then inform future queries. To facilitate this, an additional data repository is required for storing user-conversation data. Each conversation will be assigned a unique conversation ID, and with each new query, we’ll combine it with historical data associated with the conversation ID to construct a more informed query.
Conclusion
Developing a chat-based system can streamline various operations within a company. For instance, for a support team, it entails inputting all relevant informational documents, historical records, and frequently asked questions (FAQs) into the system, thereby offering a user-friendly and efficient search option for support teams. Additionally, it enables the creation of a self-help system for end users.
Consider the possibility of inputting an entire repository of Java-related information, including documents, whitepapers, ebooks, and PDFs from a university, and establishing an easily navigable search system.
The potential benefits of a chatbot-based assistant are boundless. It has the capacity to revolutionize your company and its products.