Unleashing the Power of Semantic Kernel and Azure Cognitive Search: A Step-by-Step Guide to Building Your Own ChatGPT-like App with Internal Data!— Part 2
Architecture:
Before starting, we need to understand some basic concepts about Semantic Kernel Components:
- Plugins/Skills : In the context of a semantic kernel, plugins refer to modular components or extensions that enhance the capabilities of the kernel. These plugins can be designed to perform specific tasks or provide additional functionality to the semantic analysis system. There are 2 types of plugins: Semantic and Native Plugins. In this article we will work on Semantic Plugins. This is example of Summarize skill from Semantic Kernel . Semantic plugin for me is basically “Prompt” with placeholder to “knowledgebase doc”.
- Memory: Memory, in the context of a semantic kernel, refers to the ability of the system to retain and utilize knowledge or information from previous interactions or inputs. Memory can also be used to store information that you want to be used for your skills. For example, in our case we are storing Internal Document for our “XYZ” company, so that ChatGPT.
There are advanced concepts, like Planners, Chaining of skills, etc. This is not required for our simple use case. You can learn from here:
Important Concept to Understand regarding Open AI:
- Tokens: Tokens can be thought of as pieces of words. Before the Open AI APIs processes the prompts, the input is broken down into tokens. These tokens are not cut up exactly where the words start or end — tokens can include trailing spaces and even sub-words. GPT3 model has 4K token limit. Every model has different token limit. We need to make sure our prompt and data don’t exceeds the token limit.
- Prompt Engineering: Prompt engineering is the process of crafting effective and specific instructions or questions to guide a language model’s response. It involves formulating prompts that elicit desired information or behavior, optimizing inputs to achieve desired outputs, and iteratively refining prompts based on experimentation and feedback. Learn more here
- GPT Best Practices : https://platform.openai.com/docs/guides/gpt-best-practices/six-strategies-for-getting-better-results
Coming back to our goal, now we have to create bot for our XYZ Company.
Here are steps:
Step 1: Build Semantic Kernel.
- Create Ploygot Notebook in Visual Studio Code. More info here
- Install Semantic Kernel nugets in your notebook
#r "nuget: Microsoft.SemanticKernel, 0.17.230629.1-preview"
#r "nuget: Microsoft.SemanticKernel.Connectors.Memory.AzureCognitiveSearch, 0.17.230629.1-preview"
#r "nuget: Microsoft.SemanticKernel.Core, 0.17.230629.1-preview"- Create API key from Open AI — Learn more here
- Copy your ACS instance Endpoint and Keys
- Create Kernel instance using Builder extension. We will add Azure Cognitive Search as our Semantic Memory
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Memory;
using Microsoft.SemanticKernel.Connectors.Memory.AzureCognitiveSearch;
IKernel semanticKernel = Kernel.Builder
.WithOpenAITextCompletionService(
"text-davinci-003", // OpenAI Model Name
"_PASTE_YOUR_OPENAI_KEY_HERE_" // OpenAI Key
)
.WithAzureCognitiveSearchMemory("https://<YOUR_ACS_INSTANCE>.search.windows.net", "_PASTE_YOUR_ACS_KEY_HERE_")
.Build();Note: The best practices for storing the keys, are by using EnvVariables, Azure KeyVault, etc. For this article, I am passing it in plain-text.
Step 2: Create Skill/Plugin to Summarise the document from Azure Cognitive Search. Skills can be created by one or more ways. There are two types of skills:
- Semantic Skills
- Native Skills
I am using Semantic Skill, and I am going to create the inline skill. You can also create the skill in directory, and import to kernel. Learn more here
using Microsoft.SemanticKernel.SemanticFunctions;
string skPrompt = """
Summarize the content {{$documentFromACS}}.
User: {{$input}}
""";
var promptConfig = new PromptTemplateConfig
{
Completion =
{
MaxTokens = 2000,
Temperature = 0.2,
TopP = 0.5,
}
};
var promptTemplate = new PromptTemplate(
skPrompt,
promptConfig,
semanticKernel
);// Create and Register skill with Kernel
var functionConfig = new SemanticFunctionConfig(promptConfig, promptTemplate);
var summaryFunction = semanticKernel.RegisterSemanticFunction("MySkill", "Summary", functionConfig);
Step 3: Create Chat Context:
// Create Context for chat. Context stores all relevant data for chat session.
var context = semanticKernel.CreateNewContext();Step 4: Create Chat Function
Now we have to create chat function, which will set the relevant context for the chat. You can see we are setting up the variables {{input}} and {{documentFromACS}} in context, as it will be needed to invoke Summarisation Skill.
Func<string, Task> Chat = async (string input) => {
// save the context variable input
context["input"] = input;
// Get semantic similar data from Azure Cognitive SearchAsync
var memories = semanticKernel.Memory.SearchAsync("internaldoc", input, limit: 1, minRelevanceScore: 0.5);
var internalDoc = "";
await foreach (MemoryQueryResult memory in memories)
{
Console.WriteLine("<-- Semantically Similar doc found --> " + memory.Metadata.Id);
internalDoc = memory.Metadata.Text;
}
context["documentFromACS"] = internalDoc;
// Process the user message and get an answer
var answer = await summaryFunction.InvokeAsync(context);
Console.WriteLine("<-- Document Summarize by GPT --> ");
// Show the response
Console.WriteLine(answer);
};In Part 1, we created the ACS data ingestion pipeline and it should ingest data into ACS.
When user ask Question :
- We do semantic search on Azure Cognitive Index <INDEX_NAME>. Documents with confidence level > 50% are returned (minRelevanceScore).
- All documents which are semantically similar to <USER_QUESTION> are returned. You can define limit for documents retrieval.
SearchAsync(<INDEX_NAME>, <USER_QUESTION>, limit: 1, minRelevanceScore: 0.5);
Step 4: Let’s Chat!
I have added some sample documents in ACS regarding, company ‘XYZ’ revenue, marketing strategy and funding.
// example 1
await Chat("I want to know company earning");
// example 2
await Chat("I want to XYZ Company marketing stratergy");
You can now become more creative with adding some great skills and more docs to ACS. You can expose this as API, and use with Bot Framework to create bot.
I wil be adding the sample notebook to GitHub : https://github.com/akshaykokane/ChatGPTLikeApp/tree/main
References:
