Local Memory: C# Semantic Kernel, Ollama and SQLite to manage Chat Memories locally

Note: Some of this content was written/revised using Generative Artificial Intelligence tools

This is my first post in a series where I detail my experiences with implementing Large Language Models in .NET using open-source models that can operate locally, without the need for subscriptions or API keys.

As an enthusiast of Python, I recognize its pivotal role in advancing data science and the development of Large Language Models (LLMs) through tools like LangChain and libraries within the Python ecosystem.

However, deploying Python-based tools often requires a cloud environment, which can be prohibitively expensive for an independent developer like me. My goal is to create free, open-source applications that leverage LLMs without relying on server backends, enabling deployment across various platforms, including macOS, Windows, and the web.

In this post I will show an example using Semantic Kernel and Ollama with a local SQLite database to manage memory. This example uses the Semantic Kernel Ollama plugin you can find here:

GitHub — BLaZeKiLL/Codeblaze.SemanticKernel: Some Cool Semantic Kernel Plugins

I am told this functionality is under consideration to be incorporated into Semantic Kernel in future releases.

In this example we will show how to connect Semantic Memory to Ollama and use SQLite for memory.

The completed code can be downloaded here:

https://github.com/john-c-kane/sk-ollama-memory

Step 1. Install Ollama

https://ollama.com/

Step 2. Download the all-minilm model weights

To download the LLM and embedding weights simply open a command prompt and type “ollama pull …”.

C:\>ollama pull llama3
C:\>ollama pull all-minilm

Run the following notebook in Visual Studio Code.

This code requires the following extensions:

Polyglot

C#

Visual Studio Code with the Polyglot and C# extensions installed.

//example Semantic Kernel Memory with Ollama and Sqlite connectors
//Github: based on https://github.com/BLaZeKiLL/Codeblaze.SemanticKernel
#r "nuget: Microsoft.SemanticKernel, 1.10.0"
#r "nuget: Microsoft.SemanticKernel.Plugins.Memory, *-*"
#r "nuget: Microsoft.SemanticKernel.Connectors.Sqlite, 1.10.0-alpha"
#r "nuget: System.Net.Http.Json, 8.0.0"
#r "nuget: Codeblaze.SemanticKernel.Connectors.Ollama, 1.3.0"
#r "nuget: System.Linq.Async, 6.0.1"

using System.Net.Http;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.Sqlite;
using Microsoft.SemanticKernel.Memory;
using Kernel = Microsoft.SemanticKernel.Kernel;
using Microsoft.SemanticKernel.Plugins.Memory;
using Codeblaze.SemanticKernel.Connectors.Ollama;

//instantiate kernel using openAI API with Ollama Chat URL
#pragma warning disable SKEXP0010
var builder = Kernel.CreateBuilder();
var endpoint = new Uri("http://localhost:11434");
var modelId = "llama3";

var kernelBuilder = Kernel.CreateBuilder().AddOpenAIChatCompletion( modelId: modelId, apiKey: null, endpoint: endpoint);
var kernel = kernelBuilder.Build();

#pragma warning disable SKEXP0001, SKEXP0020
var builder_memory = new MemoryBuilder();
var embeddingEndpoint = "http://localhost:11434";
var cancellationTokenSource = new System.Threading.CancellationTokenSource();
var cancellationToken = cancellationTokenSource.Token;
builder_memory.WithHttpClient(new HttpClient());
builder_memory.WithOllamaTextEmbeddingGeneration("all-minilm", embeddingEndpoint);
//builder.WithOllamaTextEmbeddingGeneration("nomic-embed-text", embeddingEndpoint);
//builder.WithOllamaTextEmbeddingGeneration("mxbai-embed-large", embeddingEndpoint);
var sqliteMemoryStore = await SqliteMemoryStore.ConnectAsync("memories.sqlite", cancellationToken);
builder_memory.WithMemoryStore(sqliteMemoryStore);
var memory = builder_memory.Build();

const string MemoryCollectionName = "aboutMe";
await memory.SaveInformationAsync(MemoryCollectionName, id: "info1", text: "My name is Andrea");
await memory.SaveInformationAsync(MemoryCollectionName, id: "info2", text: "I currently work as a tourist operator");
await memory.SaveInformationAsync(MemoryCollectionName, id: "info3", text: "I live in Seattle and have been living there since 2005");
await memory.SaveInformationAsync(MemoryCollectionName, id: "info4", text: "I visited France and Italy five times since 2015");
await memory.SaveInformationAsync(MemoryCollectionName, id: "info5", text: "My family is from New York");

var history = "";
var questions = new[]
{
    "what is my name?",
    "where do I live?",
    "where is my family from?",
    "where have I travelled?",
    "what do I do for work?",
};

foreach (var q in questions)
{
    var response = await memory.SearchAsync(MemoryCollectionName, q, limit: 1, minRelevanceScore: 0.3).FirstOrDefaultAsync();
    history += response?.Metadata.Text + "\n";
    Console.WriteLine(q + " " + response?.Metadata.Text);
}

Model Answers: what is my name? My name is Andrea where do I live? I live in Seattle and have been living there since 2005 where is my family from? My family is from New York where have I travelled? I currently work as a tourist operator what do I do for work? I currently work as a tourist operator

using Microsoft.SemanticKernel.Plugins.Memory;
using Microsoft.SemanticKernel.Connectors.OpenAI;

#pragma warning disable SKEXP0050, SKEXP0001
// TextMemoryPlugin provides the "recall" function
kernel.ImportPluginFromObject(new TextMemoryPlugin(memory));

const string skPrompt = @"
ChatBot can have a conversation with you about any topic.
It can give explicit instructions or say 'I don't know' if it does not have an answer.

Information about me, from previous conversations:
- {{$fact1}} {{recall $fact1}}
- {{$fact2}} {{recall $fact2}}
- {{$fact3}} {{recall $fact3}}
- {{$fact4}} {{recall $fact4}}
- {{$fact5}} {{recall $fact5}}

Chat:
{{$history}}
User: {{$userInput}}
ChatBot: ";

var chatFunction = kernel.CreateFunctionFromPrompt(skPrompt, new OpenAIPromptExecutionSettings { MaxTokens = 200, Temperature = 0.8 });

#pragma warning disable SKEXP0001, SKEXP0050
var arguments = new KernelArguments();
arguments["fact1"] = "what is my name?";
arguments["fact2"] = "where do I live?";
arguments["fact3"] = "where is my family from?";
arguments["fact4"] = "where have I travelled?";
arguments["fact5"] = "what do I do for work?";
arguments[TextMemoryPlugin.CollectionParam] = MemoryCollectionName;
arguments[TextMemoryPlugin.LimitParam] = "2";
arguments[TextMemoryPlugin.RelevanceParam] = "0.8";

arguments["history"] = history;
Func<string, Task> Chat = async (string input) => {
    // Save new message in the kernel arguments
    arguments["userInput"] = input;

    // Process the user message and get an answer
    var answer = await chatFunction.InvokeAsync(kernel, arguments);

    // Append the new interaction to the chat history
    var result = $"\nUser: {input}\nChatBot: {answer}\n";

    history += result;
    arguments["history"] = history;
    
    // Show the bot response
    Console.WriteLine(result);
};

//await Chat("Hello, I think we've met before, remember? my name is...");
await Chat("Tell me everything you remember from our previous conversations.");

User: Tell me everything you remember from our previous conversations. ChatBot: Let’s recap the details I’ve learned about you! From our previous conversations, I recall that: 1. Your name is Andrea. 2. You reside in Seattle and have been living there since 2005. 3. Your family is from New York. 4. You work as a tourist operator, guiding travelers around the city. That’s all the information I’ve gathered about you so far! Would you like to chat more or share any new experiences?