Introduction to Semantic Kernel Planners for Seamless Orchestration
Discover the transformative potential of building an AI Orchestrator with Semantic Kernel Planners. Explore how this cutting-edge technology empowers AI systems to seamlessly orchestrate complex tasks, unlocking new possibilities for efficiency and productivity.
[UPDATE Jan 2024] — Some of the planners mentioned here are no more available in latest versions. Check out this blog https://medium.com/@akshaykokane09/semantic-kernel-v1-x-making-ai-integration-easier-than-ever-1e1132b583e4 for latest updates.
In my previous “Unleashing the Power of Semantic Kernel and Azure Cognitive Search: A Step-by-Step Guide to Building Your Own ChatGPT-like App with Internal Data!” we saw how Semantic Kernel can be used to create Skills/Plugins and how to invoke them.
In this blog, we will explore different planner options in Semantic Kernel. In semantic kernel currently (as of Jul 9 , 2023) there are 3 types of planners.
- Action Planner
- Sequential Planner
- Stepwise Planner (preview)
What are Plugins?
Lets understand first, what are plugins. Plugins in Semantic Kernel are modular functions that developers can build to extend the capabilities of AI systems. There are two types functions within plugins:
- Semantic Functions:
Semantic functions are prompts, with context variables {{varName}} . Semantic Functions are usually text in and text out. Example is below:
Example: Translate Plugin. Every semantic plugin comes with config.json file. Config.json file is important, because it provides metadata which planner leverage for orchestration.
Translate the input below into {{$language}}
MAKE SURE YOU ONLY USE {{$language}}.
{{$input}}
Translation:{
"schema": 1,
"type": "completion",
"description": "Translate the input into a language of your choice",
"completion": {
"max_tokens": 2000,
"temperature": 0.7,
"top_p": 0.0,
"presence_penalty": 0.0,
"frequency_penalty": 0.0,
"stop_sequences": [
"[done]"
]
}
}2. Native Functions:
Native functions in an AI app are like the app’s hands. They allow the app to directly use C# or Python code to manipulate data and perform various tasks, such as saving and retrieving data or doing calculations. These functions are handy for operations that the AI model itself (LLMs) may not be well-suited for. [SKFunction, Description] is important, as planner relies on the description for orchestrating this function at correct time.
[SKFunction, Description("Read all text from a document")]
[SKFunctionInput(Description = "Path to the file to read")]
public async Task<string> ReadTextAsync(string filePath, SKContext context)
{
this._logger.LogInformation("Reading text from {0}", filePath);
using var stream = await this._fileSystemConnector.GetFileContentStreamAsync(filePath, context.CancellationToken).ConfigureAwait(false);
return this._documentConnector.ReadText(stream);
}What are Planners?
TL;DR: A planner is a function that uses AI to combine plugins and generate plans for user requests. It allows developers to create atomic functions that can be used in unforeseen ways. For example, a task and calendar plugin can be combined to create reminders. However, it’s important to be mindful of exposing only desired functions and applying responsible AI principles for fair, reliable, safe, private, and secure usage.
Note: Sometimes plugins are interchangeable with functions or skills. In previous versions of Semantic Kernel, plugins were called as skills
A planner is a powerful function that takes a user’s request and generates a plan for accomplishing that request. It utilizes AI technology to dynamically combine registered plugins in the kernel, creating a series of steps that lead to the desired goal.
This concept enables developers to create small, self-contained functions that can be utilized in unforeseen ways. For instance, by having plugins for tasks and calendar events, the planner can effortlessly create workflows such as “remind me to buy milk when I go to the store” or “remind me to call my mom tomorrow” without requiring explicit code for each scenario.
The planner automatically combines these functions, but it’s crucial to exercise caution. As it can combine functions in unexpected ways, it’s important to carefully expose only the functions intended for such usage. Additionally, responsible AI principles should be applied to ensure fairness, reliability, safety, privacy, and security in the utilization of these functions.
How planner knows which function to invoke?
In simple terms, when using Semantic and Native functions, the app relies on information we provide about the functions (metadata for Semantic and function decorators for Native).
This information helps the Semantic Kernel to understand what each function does. So, when a user asks a question, the SK uses this knowledge to find the right function(s) that can fulfill the user’s request. It’s like the app knows which tools to use based on what the user asks for.
Types of Planners:
Action Planner:
- Action Planner is a tool in the Semantic Kernel SDK that can be used to orchestrate a single plugin.
- Action Planner is best suited for simple tasks that can be completed by a single plugin. It can also be used to identify the intent of a user’s request.
- For example, you could use Action Planner to generate a weather forecast, find a restaurant, or book a flight.
- Action Planner can only be used to control one plugin at a time. If you need to control multiple plugins, you will need to use a Sequential or Stepwise Planner
Example, click here
Sequential Planner:
- A Sequential planner serves as a way to link and execute multiple functions in a step-by-step manner. Imagine it as a helpful tool when you require multiple plugins or functions to run consecutively.
- This planner allows you to pass the output of one function as the input to another, creating a seamless flow of information.
- For instance, let’s say you want to obtain a summary of the weather data for the last 7 days. Achieving this would involve using two functions. First, the ‘FetchWeatherData’ function will retrieve the weather data for the specified period. Then, the output from this function will be passed on as the input to the ‘WeatherSummarization’ function, which will process the data and present the desired summary. This way, you can efficiently chain these functions together to get the information you need in a coherent manner.
- Sequential planner takes user question, makes plan based on available functions and then invoke plan i.e executes functions one after the other as per the plan
- If you want to know, how it works internally, I suggest reading this doc
- Example: We want to convert the text to UPPER CASE and append DATE to answer: We have 3 native functions https://github.com/akshaykokane/ChatGPTLikeApp-SemanticKernelOpenAI/blob/main/SequentialPlannerEx.ipynb
// Copyright (c) Microsoft. All rights reserved.
// I am copying directily from https://github.com/microsoft/semantic-kernel two basic native function
using System.ComponentModel;
using Microsoft.SemanticKernel.SkillDefinition;
public sealed class StaticTextSkill
{
[SKFunction, Description("Change all string chars to uppercase")]
public static string Uppercase([Description("Text to uppercase")] string input) =>
input.ToUpperInvariant();
[SKFunction, Description("Append the day variable")]
public static string AppendDay(
[Description("Text to append to")] string input,
[Description("Value of the day to append")] string day) =>
input + day;
[SKFunction, Description("Get today's day")]
public static string GetTodayDay() =>
DateTime.Today.DayOfWeek.ToString();
}
// register native function
semanticKernel.ImportSkill(new StaticTextSkill(), "StaticTextSkill"); using Microsoft.SemanticKernel.Planning;
using Microsoft.SemanticKernel.Planning.Sequential;
var plannerConfig = new SequentialPlannerConfig();
plannerConfig.MaxTokens = 1000;
SequentialPlanner planner = new SequentialPlanner(semanticKernel, plannerConfig);
var plan = await planner.CreatePlanAsync("convert this text to upper case and append today's day : Hi, today is lovely day : ");
var result = await plan.InvokeAsync(semanticKernel.CreateNewContext()); Plan: {
"state": [
{
"Key": "RESULT__FINAL_ANSWER",
"Value": "HI, TODAY IS LOVELY DAY :SATURDAY"
},
{
"Key": "PLAN.RESULT",
"Value": "\nHI, TODAY IS LOVELY DAY :SATURDAY"
},
{
"Key": "TODAY_DAY",
"Value": "Saturday"
},
{
"Key": "INPUT_WITH_DAY",
"Value": "Hi, today is lovely day :Saturday"
},
{
"Key": "INPUT",
"Value": "HI, TODAY IS LOVELY DAY :SATURDAY"
}
],
"steps": [
{
"state": [
{
"Key": "INPUT",
"Value": ""
}
],
"steps": [],
"parameters": [
{
"Key": "INPUT",
"Value": ""
}
],
"outputs": [
"TODAY_DAY"
],
"next_step_index": 0,
"name": "GetTodayDay",
"skill_name": "StaticTextSkill",
"description": "Get today\u0027s day"
},
{
"state": [
{
"Key": "INPUT",
"Value": ""
}
],
"steps": [],
"parameters": [
{
"Key": "day",
"Value": "$TODAY_DAY"
},
{
"Key": "INPUT",
"Value": "Hi, today is lovely day :"
}
],
"outputs": [
"INPUT_WITH_DAY"
],
"next_step_index": 0,
"name": "AppendDay",
"skill_name": "StaticTextSkill",
"description": "Append the day variable"
},
{
"state": [
{
"Key": "INPUT",
"Value": ""
}
],
"steps": [],
"parameters": [
{
"Key": "INPUT",
"Value": "$INPUT_WITH_DAY"
}
],
"outputs": [
"RESULT__FINAL_ANSWER"
],
"next_step_index": 0,
"name": "Uppercase",
"skill_name": "StaticTextSkill",
"description": "Change all string chars to uppercase"
}
],
"parameters": [
{
"Key": "INPUT",
"Value": ""
}
],
"outputs": [
"RESULT__FINAL_ANSWER"
],
"next_step_index": 3,
"name": "",
"skill_name": "Microsoft.SemanticKernel.Planning.Plan",
"description": "convert this text to upper case and append today\u0027s day : Hi, today is lovely day : "
}Final Answer is :
HI, TODAY IS LOVELY DAY :SATURDAYMore Example, click here
Stepwise Planner:
- The stepwise planner is an advanced version of the sequential planner. Unlike the sequential planner, it doesn’t pre-build the entire plan and execute functions in a predefined order. Instead, it generates the plan dynamically as it goes along.
- Stepwise Planner is based on MRKL System and can be used where you need planner to think like Human before making any decision.
- Here’s how it works: At each step, it decides which action to take based on the output of the previous step, followed by generating a corresponding thought process and observation. In essence, it reasons and evaluates after every step. The output of each step guides which function to invoke next and what information to pass to the subsequent step.
- This type of planner is particularly useful for handling more complex tasks that require making decisions based on intermediate outputs. It ensures that every [ACTION] is followed by a [THOUGHT] process and [OBSERVATION], continuing this cycle until it eventually reaches the [FINAL ANSWER].
- For example: https://github.com/akshaykokane/ChatGPTLikeApp-SemanticKernelOpenAI/blob/main/StepwisePlannerEx.ipynb
using Microsoft.SemanticKernel.Planning;
using Microsoft.SemanticKernel.Planning.Stepwise;
var plannerConfig = new Microsoft.SemanticKernel.Planning.Stepwise.StepwisePlannerConfig();
plannerConfig.MinIterationTimeMs = 1500;
plannerConfig.MaxTokens = 1000;
StepwisePlanner planner = new StepwisePlanner(semanticKernel, plannerConfig);
var plan = planner.CreatePlan("convert this text to upper case and append today's day : Hi, today is lovely day : ");
var result = await plan.InvokeAsync(semanticKernel.CreateNewContext());Steps Taken: 5
This was my previous work (but they haven't seen any of it! They only see what I return as final answer):
[THOUGHT]
[OBSERVATION] System step error, no thought or action found. Please give a valid thought and/or action.
[THOUGHT] I need to use the StaticTextSkill.Uppercase and StaticTextSkill.AppendDay functions to convert the text to uppercase and append today's day.
[ACTION] {"action": "StaticTextSkill.Uppercase","action_variables": {"input":"Hi, today is lovely day"}}
[OBSERVATION] HI, TODAY IS LOVELY DAY
[THOUGHT] [ACTION] {"action": "StaticTextSkill.GetTodayDay"}
[ACTION] {"action": "StaticTextSkill.GetTodayDay","action_variables": null}
[OBSERVATION] Saturday
[THOUGHT] I need to use the StaticTextSkill.AppendDay function to append the day to the text.
[ACTION] {"action": "StaticTextSkill.AppendDay","action_variables": {"input":"HI, TODAY IS LOVELY DAY","day":"Saturday"}}
[OBSERVATION] HI, TODAY IS LOVELY DAYSaturday
[THOUGHT]
Final Answer is : The text "Hi, today is lovely day" converted to uppercase and appended with today's day (Saturday) is "HI, TODAY IS LOVELY DAYSaturday".More Example, click here
…I anticipate that this list will continue to expand.
Conclusion:
Your choice of planner depends on the intended use. Personally, I favor the sequential planner over the stepwise one, as the latter tends to take more time to provide answers. Furthermore, using a stepwise planner increases the likelihood of hallucination, where the LLM model generates [OBSERVATION] and generates [THOUGHT] at each step of the plan
References:
