Introduction
In the ever-evolving landscape of enterprise solutions, ServiceNow has emerged as a transformative platform that streamlines business operations, empowers IT departments, and enhances user experiences across organizations. However, as the digital era unfolds, the sheer volume of data and documents within ServiceNow can often be overwhelming. Finding the right information at the right time has become a crucial challenge.
Imagine a world where you can seamlessly interact with your ServiceNow documents through natural language conversations, much like chatting with a trusted colleague. Picture a scenario where ServiceNow users can effortlessly retrieve relevant documents, glean insights, and make informed decisions without wrestling with complex search queries or sifting through folders.
This is where the fusion of OpenAI Embedding and Semantic Search steps in to revolutionize the ServiceNow experience. In this article, we’ll embark on a journey to discover how integrating OpenAI’s cutting-edge embedding technology and semantic search capabilities can unlock the full potential of your ServiceNow platform. We’ll explore the power of vector data, delve into the intricacies of semantic search, and provide you with a comprehensive guide on implementing this game-changing technology.
Table of Contents
- Understanding the Terminologies
- Roadmap of the Project
- High-level steps for Implementation
- Getting Started with Implementation
- The Future of AI in ServiceNow
- Conclusion
Understanding the Terminologies
Lets understand few terminologies in very layman’s terms before we move ahead with implementation.
Embedding : Embedding is like giving each word a special number so that a computer can understand and work with words more easily. These numbers, help the computer compare words, find similarities, and do language-related tasks faster and more accurately. It’s like creating a secret language for computers to understand words better.
Vector Store : A vector store is like a special container(database) where we keep information about words and it’s equivalent embedding. Imagine each word having its own set of numbers that describe it uniquely. These numbers are like a secret code that helps computers understand what words mean and how they relate to each other. So, when you ask a computer a question or want it to find something for you, it can use these special codes to figure things out faster and better, kind of like how a treasure map helps you find hidden treasure.
Semantic Search : A semantic search is like a super-smart search engine that understands the meaning of words and phrases, not just the exact words you type in. It’s like, if you ask it questions, and it will find things that make sense, even if the words are a little different. For example, if you’re looking for information about cute puppies, it won’t just find context with the word “puppies,” but also context about adorable dogs because it knows they’re related. It’s like having a search buddy that understands what you really mean, not just the words you use, so you can find the most helpful information easily.
Roadmap of the Project
Before we start take a look of this Flow Diagram.
High-level steps for Implementation
Lets analyze the flow and break it down in steps. Please note, you can use any other tools/providers to achieve same desired result, as per your wish:
Step 1: Getting the Document File
First step is to get the document file from the user. I will be utilizing ‘File Picker’ component of ServiceNow’s Virtual Agent. I have only covered for PDFs in this implementation.
Step 2: Retrieve text from the Document
Next, we will retrieve all the text from the document. I have used a third party API provider ConvertAPI to extract text from PDF.
Step 3: Split the text in smaller chunks
We will then split huge text from the PDF document into smaller chunks. This splitting is required because we only need context which are related to user question and the LLM models (like gpt-3.5 turbo) has some character limits.
Step 4: Create embedding of all the chunks
After this , We will embed all the chunks into vectors. For this purpose I utilized OpenAI Embeddings. OpenAI’s text embeddings are really powerful and fast.
Step 5: Store embeddings in a vector database
Next, We will store the chunks and it’s equivalent embeddings in a vector database. I have used Qdrant for this purpose. You can also use Pinecone.
Step 6: Perform semantic search and retrieve similar chunks
Now we will take user question and embed it using same OpenAI’s text embeddings . Then we will perform a semantic search in our vector database to retrieve top 3 text chunks similar to user’s question with highest ranked result. The combination of these chunks will be our Context.
Step 7: Pass user question to LLM model with Context
We then pass user’s question with the Context we retrieved in the previous step to an LLM model (OpenAI’s gpt-3.5-turbo in our case).
Step 8: Get the result from LLM and show it to the user
Finally, we will then display the result from gpt to the user.
Getting Started with Implementation
Let’s start with setting up pre-requisites for this project
- Create an OpenAI API key from here.
- Create an account in ConvertAPI from here. Goto Authentication in account and get API Secret.
3. Create an account in Qdrant from here. After you login to Qdrant account create a Cluster and generate an API Key for access. Also copy your Qdrant endpoint.
Then you need to create a collection in which you can store points(vectors).
To create a collection, I used Postman with the api key and endpoint of Qdrant. This is required to be done only once.
Create a PUT HTTP method in Postman and under Authorization select Type as API Key, then enter Key as ‘api-key’ and Value as <<your_qdrant_api_key>>.
Then enter your Qdrant endpoint and append it with /collections/<<your_collection_name>>. I am keeping my collection name as rk-collection.
Under Body tab select ‘raw’ and enter the body as below:
{
"vectors": {
"size": 1536,
"distance": "Cosine"
}
}NOTE: size = 1536 only if you are using OpenAI’s text embeddings. If you are using any other platform for text embedding, you need to check for its size.
Now click on ‘Send’ and it will create a collection in Qdrant. You can verify it by going into your Qdrant Account and then Dashboard.
Now we are good to Implement our project in ServiceNow.
Implementation in ServiceNow
Creating outbound REST Messages
First we will create a couple of Outbound REST messages in ServiceNow. For this we will go to System Web Services -> Outbound -> REST Message.
OpenAI
Endpoint : https://api.openai.com/v1/chat/completions
Under HTTP Request we will have 2 Headers:
Name : Authorization , Value : Bearer <<your_openai_api_key>>
Name : Content-Type, Value : application/json
In the related list we will create 2 HTTP methods
chat_with_document
HTTP method : POST
Endpoint : <<keep_this_empty>>
Content :
{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "system",
"content": "I am providing you context and question delimited by triple hash '###'. Please give me answer with relation to the context. Do the analysis if required"
},
{
"role": "user",
"content": "${question}"
}
]
}gpt_embeddings
HTTP method : POST
Endpoint : https://api.openai.com/v1/embeddings
Content :
{
"input": ${arrayOfTexts},
"model": "text-embedding-ada-002"
}Completed REST message for OpenAI. We will now create another REST message for ConvertAPI.
ConvertAPI
Endpoint : https://v2.convertapi.com/convert/pdf/to/txt
Under HTTP Request we will have 1 Header:
Name : Content-Type, Value : application/json
In the related list we will create 1 HTTP method
convert_pdf_to_text
HTTP method : POST
Endpoint : <<keep_this_empty>>
HTTP Query Parameters :
Name : Secret, Value : <<your_convertapi_api_key>>
Content :
{
"Parameters": [
{
"Name": "File",
"FileValue": {
"Name": "${nameOfFile}",
"Data": "${base64ofAttachment}"
}
},
{
"Name": "FileName",
"Value": "output"
}
]
}
Completed REST message for ConvertAPI. We will now create another REST message for Qdrant.
Qdrant
Endpoint : <<your_qdrant_endpoint>>
We will provide endpoint in each HTTP method.
Under HTTP Request we will have 2 Headers:
Name : api-key, Value : <<your_qdrant_api_key>>
Name : Content-Type, Value : application/json
In the related list we will create 3 HTTP methods
insert_point
HTTP method : PUT
Endpoint : <<your_qdrant_endpoint>>/collections/<<your_collection_name>>/points?wait=true
Content :
{
"points": ${arrOfPoints}
}
semantic_search
HTTP method : POST
Endpoint : <<your_qdrant_endpoint>>/collections/<<your_collection_name>>/points/search
Content :
{
"vector": ${inputVectorArr},
"params": {
"hnsw_ef": 0,
"exact": false,
"quantization": null
},
"limit": 3,
"offset": 0,
"with_payload": true,
"with_vector": true,
"score_threshold": 0
}
delete_all_points
HTTP method : POST
Endpoint : <<your_qdrant_endpoint>>/collections/<<your_collection_name>>/points/delete
Content :
{
"filter": {}
}
And we are done with creating all the required Outbound REST messages.
Creating Script Include and required functions
First we will create a property in sys_properties table in ServiceNow which will store the base64 encoding of attachment. We are doing this to verify that same file is not embedded again and again.
Property name : gpt.attachment.embeded
Type : string
Now we will create a Script Include in ServiceNow and a couple of functions inside it.
Script Include Name : gptUtils
Client callable : Checked
First Function : embedDocument
This function will run after user upload PDF. This will extract the text from PDF. Split it in chunks. Embed it and then store it in Qdrant collection.
embedDocument: function(sysAttID) {
//Extract Base64 of the attachment
var attachmentIS = new GlideSysAttachmentInputStream(sysAttID);
var byteArrayOS = new Packages.java.io.ByteArrayOutputStream();
attachmentIS.writeTo(byteArrayOS);
b64attachment = GlideBase64.encode(byteArrayOS.toByteArray());
//Send Attachment to ConvertAPI and get fullText
var rawText = new fileConversionUtils().pdfToText(sysAttID);
var fullText = rawText.replaceAll(/\s+/g, ' ').trim();
//parse full_text into chunks array using my own function
var textChunks = split_in_chunks(fullText);
//Send chunks to OpenAI and get the Index and embeddings
if (gs.getProperty('gpt.attachment.embeded') != b64attachment) {
gs.setProperty('gpt.attachment.embeded', b64attachment);
var qdrantPointsArr = createPayloadForQdrant(textChunks);
//Clean the collection first
var rDelQdrant = new sn_ws.RESTMessageV2('Qdrant', 'delete_all_points');
response = rDelQdrant.execute();
//Send Chunk text and Embeddings to QdrantAPI
var rQdrant = new sn_ws.RESTMessageV2('Qdrant', 'insert_point');
rQdrant.setStringParameterNoEscape('arrOfPoints', JSON.stringify(qdrantPointsArr));
response = rQdrant.execute();
responseBody = JSON.parse(response.getBody());
if (responseBody.status == 'ok') {
return "Successfully Embeded and stored in Vector Database ! Please continue with the Chat";
} else {
return "Error on Building Points : " + JSON.stringify(responseBody);
}
}else{
return "This file is already Embeded ! Please continue with the Chat";
}
//====================================
//Functions Section
//====================================
//Function to Embed Text Chunks and return a Qdrant Payload
function createPayloadForQdrant(textChunks) {
var rEmbed = new sn_ws.RESTMessageV2('OpenAI', 'gpt_embeddings');
rEmbed.setStringParameterNoEscape('arrayOfTexts', JSON.stringify(textChunks));
response = rEmbed.execute();
responseBody = JSON.parse(response.getBody());
var arrayOfEmbeddings = responseBody.data;
var qdrantPointsArr = [];
//Prepare Qdrant Payload(Array of Objects)
arrayOfEmbeddings.forEach(function(embed) {
qdrantPointsArr.push({
"id": embed.index,
"vector": embed.embedding,
"payload": {
"text": textChunks[embed.index]
}
});
});
return qdrantPointsArr;
}
//Function for spliting text in small chunks
function split_in_chunks(text) {
var chunks = [];
var start = 0;
var data_size = 512;
var end = start + data_size;
var actualEnd = end;
var chunk_size;
while (end <= text.length()) {
chunk_size = 100; //its actually chunk_overlap
if (String.fromCharCode(text.charAt(end - 1)) === '.') {
chunks.push(text.substring(start, end).trim());
} else {
while (end > start && String.fromCharCode(text.charAt(end - 1)) !== '.' && chunk_size > 0) {
end--;
chunk_size--;
}
chunks.push(text.substring(start, actualEnd).trim());
}
start = end;
end = start + data_size;
actualEnd = end;
}
chunks.push(text.substring(start, text.length()).trim());
return chunks;
}
},Second Function : chatWithDocument
This function will be used to do a semantic search in our Qdrant database and pass the question with context to gpt. This will return the final answer.
chatWithDocument: function(prompt){
var promptInArr = '["' + prompt + '"]';
//Embed prompt into vector using OpenAI embeddings
var rEmbed = new sn_ws.RESTMessageV2('OpenAI', 'gpt_embeddings');
rEmbed.setStringParameterNoEscape('arrayOfTexts', promptInArr);
response = rEmbed.execute();
responseBody = JSON.parse(response.getBody());
var promptEmbedVector = responseBody.data[0].embedding;
//Search for similarity in Qdrant Vector Database and get the contexts
var rSearch = new sn_ws.RESTMessageV2('Qdrant', 'semantic_search');
rSearch.setStringParameterNoEscape('inputVectorArr', JSON.stringify(promptEmbedVector));
response = rSearch.execute();
responseBody = JSON.parse(response.getBody());
var chunks = [];
var similarChunkPaylodArr = responseBody.result;
similarChunkPaylodArr.forEach(function (payload){
chunks.push(payload.payload.text);
});
var bigMergeChunkContext = "";
chunks.forEach(function (chunk){
bigMergeChunkContext += chunk + " --- ";
});
var rawPromptWithContext = "###Context: " + bigMergeChunkContext + "### " + "###Question: " + prompt + " ###" + "Answer: ";
var promptWithContext = rawPromptWithContext.replaceAll(/\s+/g, ' ').trim();
promptWithContext = promptWithContext.replaceAll("\n", " ");
promptWithContext = promptWithContext.replaceAll("\r", " ");
var r = new sn_ws.RESTMessageV2('OpenAI', 'chat_with_document');
r.setStringParameterNoEscape('question', promptWithContext);
response = r.execute();
responseBody = JSON.parse(response.getBody());
var gptReply = responseBody.choices[0].message.content;
return gptReply;
},Script Include Name : fileConversionUtils
Client callable : Checked
Function name : pdfToText
This function will call the ConvertAPI and extract text from a PDF File.
pdfToText: function(attSysId) {
var grAtt = new GlideRecord('sys_attachment');
if (grAtt.get(attSysId)) {
var fileName = grAtt.getValue('file_name');
var gsa = new GlideSysAttachment();
var attachmentContentBase64 = GlideBase64.encode(gsa.getBytes(grAtt));
}
var r = new sn_ws.RESTMessageV2('ConvertAPI', 'convert_pdf_to_text');
r.setStringParameterNoEscape('nameOfFile', fileName);
r.setStringParameterNoEscape('base64ofAttachment', attachmentContentBase64);
response = r.execute();
responseBody = JSON.parse(response.getBody());
var responsebase64 = responseBody.Files[0].FileData;
var text = GlideStringUtil.base64Decode(responsebase64);
return text;
},Done with Script Include.
Implementing together in Virtual Agent
Go to Virtual Agent Designer. Create a new topic (lets say ‘GPT File Chat’). Insert a ‘File Picker’ Component after start. Set ‘Allow user to upload’ to ‘all file types’ in File Picker properties. Then add a script component after File Picker.
In the bottom left part , insert an input Variable called ‘user_input’ and 2 Script Variable called ‘embedOutput’ and ‘gptOutput’. After this we will write the script in Bot response ‘Script’ Component , under ‘Script response message’
(function execute() {
//Get the sysID of Attachement Record
var attachmentID;
var attGR = new GlideRecord('sys_attachment');
attGR.addQuery('table_name', 'sys_cs_conversation_task');
attGR.addQuery('sys_created_by', gs.getUserName());
attGR.orderByDesc('sys_created_on');
attGR.setLimit(1);
attGR.query();
if(attGR.next()){
attachmentID = attGR.getUniqueValue();
}
//Call Script include function to extract text, embed and store embeddings to Vector Database (Qdrant)
var replyFromEmbedScript = new gptUtils().embedDocument(attachmentID);
vaVars.embedOutput = replyFromEmbedScript;
})()Then add a bot response ‘text’ component and respond it with result returned in previous script i.e. script output embedOutput
Append the flow logic as per below screenshot:
Create a User Input Component then a Decision Utility Component. User Input will get the question form the User then Decision Utility will decide whether the user inputs anything other that ‘exit’. If yes then continue the flow else exit the chat.
If user inputs anything other than ‘exit’ then continue the flow and create another Bot Response ‘Script’ component. Write the script in ‘Script response message’ to do a semantic search and return the answer from GPT.
(function execute() {
var replyFromGPT = new gptUtils().chatWithDocument(vaInputs.user_input);
vaVars.gptOutput = replyFromGPT;
})()Here ‘user_input’ is the input variable we created earlier. This should contain the user input from input component. We are storing the reply form GPT to ‘gptOutput’ Script Variable.
Create another Bot Response ‘Text’ Component and attach the reply from GPT Variable (gptOutput) in Response message.
Join it to the User Input so the chat will go on again after the reply.
That is pretty much all. Congrats !! Now you have a ChatGPT chat bot in ServiceNow that will analyze your document and reply based on it.
The Future of AI in ServiceNow
The future of AI in ServiceNow, particularly with the integration of OpenAI’s ChatGPT API, promises a paradigm shift in user interactions and workflow management. By leveraging ChatGPT’s advanced natural language processing capabilities, ServiceNow can offer more intuitive and human-like interactions, enhancing the user experience and simplifying complex tasks. AI-powered virtual agents can now understand and respond to queries in a more contextually aware manner, driving efficiency and productivity across various service processes. With ChatGPT API, ServiceNow is well-positioned to lead the way in AI-driven service management, delivering more intelligent and personalized solutions for businesses and their customers.
Conclusion
In a world where time is of the essence, and the volume of data is overwhelming, the OpenAI Embedding and Semantic Search integration offers a game-changing solution. It empower users to extract insights and information swiftly, ultimately leading to improved productivity, better decision-making, and enhanced customer satisfaction. As we embark on this exciting journey of revolutionizing the ServiceNow experience, the possibilities are boundless, and the future looks brighter than ever. This small implementation is just the starting of a lot more innovative ideas. Think if you can implement it in more advance way like detecting fraud in invoices, analyzing graphs data from reports and responding to work accordingly. There are lot more possibilities to explore 😉.
