Semantic Kernel: The New Way to Create Artificial Intelligence Applications
In this article, we will explore Semantic Kernel, a new Microsoft SDK that simplifies the integration of AI into conventional applications.
Semantic Kernel enables developers to easily blend cutting-edge AI with native code, opening up a world of new possibilities for AI applications.
This article could go on to discuss the features and benefits of Semantic Kernel, as well as some examples of how it is currently being used.
Components
When developing a solution using Semantic Kernel, there are a series of components that we can employ to provide a better experience in our application.
Not all of them are mandatory to use, although it is advisable to be familiar with them. Those components that conform the Semantic Kernel are listed below:
- Kernel
- Memories
- Planner
- Connectors
- Plugins (aka Skills)
Kernel
Its own name already gives us a clue about the importance it holds within the SDK.
It’s in the kernel where we’ll register all connectors and plugins, in addition to configuring what’s necessary to run our program.
Furthermore, we can also add support for logs and telemetry to check the status and performance of our program, as well as to assist in debugging if necessary.
Memories
We now come to the component that allows us to provide context to user questions. This means that our Plugin can recall past conversations with the user to give context to the question they are asking. These Memories can be implemented in three different ways:
- Key-value pairs: Stored as if they were environment variables, and the search is done conventionally, meaning there must be a 1-to-1 relationship between the key and the text entered by the user.
- Local storage: Saved in a file in the local file system. When the key-value storage reaches a critical size, it’s time to switch to this type of storage.
- Semantic in-memory search: We can represent the information as vectors of numbers called embeddings.
Planner
A planner is a function that takes a user’s prompt and returns an execution plan to carry out the request. We can choose from a set of predefined planners that come with the SDK:
SequentialPlanner: Creates a plan with multiple functions that are interconnected with their own input and output variables for each function.BasicPlanner: A simpler version of SequentialPlanner that chains functions together.ActionPlanner: Creates a plan with a single function.StepwisePlanner: Executes each step incrementally, observing the results before executing the next step.
Connectors
Connectors play a very important role within Semantic Kernel as they act as a bridge between different components, enabling the exchange of information between them.
These connectors can be developed to interact with external systems, such as interacting with HuggingFace models or using the SQLite database as memory for our development.
The Semantic Kernel SDK comes with some predefined connectors that can be grouped into two areas:
Integration with AI models:
- HuggingFace
Microsoft.SemanticKernel.Connectors.AI.HuggingFace - Oobabooga
Microsoft.SemanticKernel.Connectors.AI.Oobabooga - OpenAI
Microsoft.SemanticKernel.Connectors.AO.OpenAI
Support for existing RDBMS and NoSQL Memories:
- Azure Cognitive Search
- Chroma
- DuckDB
- Kusto
- Milvus
- Pinecone
- Postgres
- Qdrant
- Redis
- SQLite
- Weaviate
Plugins
A plugin can be described as a set of functions, whether native or semantic, exposed to AI services and applications.
It’s not enough to write the code for these functions; we must also be capable of providing semantic details about their behavior, input and output parameters, as well as any potential side effects.
At this point, we need to distinguish between two types of functions:
- Semantic Functions: These functions listen to user requests and provide responses using natural language. To accomplish this, semantic functions require connectors to fulfill their purpose.
- Native Functions: These functions are written in C#, Python, and Java (currently in an experimental phase). They handle operations where AI models are not suitable, such as:
- Mathematical calculations.
- Reading and saving data from memory.
- Accessing REST APIs.
Instead of configuration files, planners can use annotations in the code of native functions to understand their operation.
Semantic Functions
With just two files, we can define a fully functional semantic function: one for configuring the function and another to define the prompt.
Let’s create a function to convert a JSON document into a data structure for a given programming language.
The function’s configuration, the JSON file, would look like this:
As we can see, there are three main sections in the configuration. The first part is where we define the function’s name and description, which is important because the Kernel will later use them to determine when to use our function.
Next, we have the completion section, where we specify the operation of the LLM model, such as the number of tokens (similar to the word count) or the temperature, which determines how imaginative we want the model to be when generating the output.
Finally, we have the input section, where we specify the parameters our function needs to work.
Now comes the part where we define the prompt for our function, and we do that in a text file named skprompt.txt.
As we can see, we use string interpolation to include the input parameters we have defined in our configuration file using the variable names defined in the “name” property.
To prevent Prompt Injection techniques, it is advisable to delimit the content with which we want our model to work to avoid unexpected results.
Native functions
As we mentioned before, a native function is one that performs a more rudimentary task, like the one in this example, that only calculates the number of lines in a source code file.
To expose this native function to the Kernel, we use decorators in the function:
SKFunction: Used to provide a description of the functionality.SKParameter: To give an idea of the usage of each of the function’s parameters (in case there is more than one).Description: Similar toSKParameterbut when there’s only one parameter.
Making it all work
Now that we’ve created our functions, both semantic and native, it’s time to make all of this work. So, we open our favorite C# code editor on Mac and create a file called Program.cs, and its content could be as follows:
The first thing we do is import a bunch of necessary namespaces to make everything work (lines 1 to 8).
Then, we create an instance of the builder (by the pattern, not because it’s a constructor 😜) that will help shape our Kernel.
var builder = KernelBuilder();Do we want to use AI models from Microsoft in Azure, OpenAI, and one that we like from the many available in Huggingface?
Well, no problem, we can include them in our kernel, as we see in lines 13 (OpenAI), 19 (Huggingface), and 24 (Azure AI).
Important notice: we will need a user and an API key for these services, and be sure to check the price of making calls to these services.
Do you need to know what’s happening at each moment? Of course! Let’s add a log to our kernel. You can see how it’s done in line 30.
Don’t reinvent the wheel; remember that Microsoft provides us with a series of plugins that we can use in our kernel, as you can see in line 43.
And finally, the time has come to import the Plugin that we’ve worked so hard on (to be honest, not really; it’s a pretty simple plugin 🤪).
To do that, first, we import the semantic functions in lines 46 and 47. What we do is pass the folder containing all our functions, and the kernel takes care of processing them and learning what each of them does.
With this, we have our Kernel with our Plugin ready to be used.
Okay, now let me tell you how to use it
Let’s imagine that we want our model to tell us the number of lines in a code file.
Well, it turns out we have a native function that does all that, so we tell the kernel that we want to use it. Remember that the function requires the file path as a parameter, which is the first thing we specify in the RunAsync function. Then, we tell the Kernel which function is responsible for running it.
The output will be something like this:
In case we want to test one of the semantic functions, we can do that as well, although the code varies a bit. We will use the semantic function for programming language conversion.
To pass the parameters to the semantic function, we use a ContextVariables type in which we provide the data to the function. In our case, this includes the source language, the code we want to translate, and the target language.
Planners: Making our Plugins Smarter
And you’re right; so far, we’ve seen how to directly use the functions, but the users of our Plugin don’t need to know or shouldn’t know the names of our functions or their parameters. This is where Planners come into play.
As you can see, what is passed to the planner is a prompt, which can well be the text that the user provides in our application.
The planner is responsible for understanding what the user wants and finding the necessary functions within our Plugin to perform the task.
Testing the planners with Prompt Flow
Tests with our planners are conducted using a VS Code extension called Prompt Flow.
It’s a visual editor where we can define functions, their input and output parameters, and also monitor their performance.
Packages
The are some packages included in Microsoft.SemanticKernel:
- Microsoft.SemanticKernel.Abstractions: contains common interfaces and classes used by the core and other SK components.
- Microsoft.SemanticKernel.Core: contains the core logic of SK, such as prompt engineering, semantic memory, and semantic functions definition and orchestration.
- Microsoft.SemanticKernel.Connectors.AI.OpenAI: connectors to OpenAI and Azure OpenAI, allowing to run semantic functions, chats, and image generation with GPT3, GPT3.5, GPT4, DALL-E2.
Other packages available at nuget.org:
- Microsoft.SemanticKernel.Connectors.Memory.Qdrant: Qdrant connector for plugins and semantic memory.
- Microsoft.SemanticKernel.Connectors.Memory.Sqlite: SQLite connector for plugins and semantic memory
- Microsoft.SemanticKernel.Plugins.Document: Document Plugin: Word processing, OpenXML, etc.
- Microsoft.SemanticKernel.Plugins.MsGraph: Microsoft Graph Plugin: access your tenant data, schedule meetings, send emails, etc.
- Microsoft.SemanticKernel.Plugins.OpenAPI: OpenAPI Plugin.
- Microsoft.SemanticKernel.Plugins.Web: Web Plugin: search the web, download files, etc.
- Microsoft.SemanticKernel.Reliability.Polly: Extension for http resiliency.
Source Code
You can take a look at the source code accompanying this article in this GitHub repository.
Conclusion
With this new framework, Microsoft lays the groundwork for the future integration of Artificial Intelligence into our developments, both on the desktop, for example, in cross-platform applications with .NET MAUI, and on the server, with ASP.NET Core.
And perhaps most importantly, organizations have a way to integrate the new AI services into their applications using their corporate data, always ensuring privacy compliance with that data.
