MemGPT: Assessing the Extent of Unlimited Context for LLMs
Three weeks ago, I came across a video discussing MemGPT on a tech channel. I was impressed with the concept and capability of MemGPT in addressing a significant limitation in the LLM field — the token limit. In this article, let’s delve into whether it lives up to the hype and is as powerful as people claim.
1. Installing MemGPT
To install MemGPT, you can use the following pip command:
pip install pymemgptNext, configure the settings by opening the command prompt (cmd) and executing the provided command.
> memgpt configureThen, input the required information, including the LLM model you intend to use, open AI API key, embedding method etc.
2. Hello World
Okay, after entering all the required information mentioned above, you can now use MemGPT by running the following command in the command prompt (cmd):
> memgpt run
As a result, you can observe that after entering my name and interests, the bot intelligently recognizes the significance of this information and updates it in its memory.
3. Testing MemGPT Basics
MemGPT is like a super-smart brain with endless memory — no limits! To test it out:
- Step 1. Ask a tricky question.
- Step 2. Give the right answer and ask MemGPT to remember.
- Step 3. Toss in some super-long sentences.
- Step 4. Ask the same question again to see if it remembers the answer from step 2.
Step 1. Ask a tricky question
Questions related to Data Engineering skills. I requested guidance on using a function in the Teradata Database from the model. Quite tricky! :)
Not surprised at all when the response from the model is completely incorrect, as this question is quite challenging and there isn’t much documentation or information available on the internet about it.
Step 2. Provide right answer
Provide detailed information on how to use this function, including the syntax.
Here’s MemGPT’s response. To ensure MemGPT remembers the content I provided, I use the command /save in the message.
Step 3. Toss in some super-long sentences
In this step, I supplied MemGPT with over 30 different pieces of information, ensuring that the total tokens in these documents exceed 20,000 tokens. The model I’m using is `gpt-3.5-turbo`, which has a token limit of 4096.
Step 4 : Ask the same question again
Here is MemGPT’s response when I repeated the question from step 1. MemGPT provided an incorrect and seemingly unrelated answer compared to the information I provided in step 2.
Lets ask MemGPT answer the question based on MemGPT’s memory.
Wow! When I asked MemGPT to answer a question based on memory, it loaded the exact information I provided before. Great!
4. Conclusion
I turned to MemGPT while attempting to build a chatbot capable of handling complex tasks like answering uncommon questions. I had previously tried solving this challenge with RAG and fine-tuning and hoped MemGPT would provide an alternative solution. The idea is, instead of preparing extensive fine-tuning data or data for the RAG system, users can interact with the chatbot, teaching it what it doesn’t know. While using MemGPT seems like a possible approach, it’s not a perfect solution as MemGPT doesn’t always know when to use information stored in memory or when to provide a direct answer without relying on memorized information. Some customization in the source code might improve its functionality.
I’ve only recently started using MemGPT, so my approach might be incorrect. Please correct me if there’s anything you find nonsensical.
