Part 1.4 : Langchain Memory
Hi, I am Mine, incase you missed previous parts, here is a little brief about what we’ve done so far; recently, I was working on a project to build a question-answering model for giving responses to the questions over the data that we have from one of our internal projects, which we use as a project management tool. I highly recommend checking the previous parts, especially Part 3. The links are below.
We saw how AWS Canvas works in Part 1. We tried Langchain to connect the model to our CSV data in Part 2. Later, the approach changed for solution, and we started to use ChatGPT-4 as the model. We created a question-answering model over multiple CSV files.
Now, we will add a memory to the conversation. If you remember, I wrote about Langchain’s modules and what they are doing on Part 2 but I want to list them again just in case.
- Model I/O : Interface with language models.
- Retrieval : Interface with application-specific data.
- Chains : Construct sequences of calls.
- Agents : Let chains choose which tools to use given high-level directives.
- Memory : Persist application state between runs of a chain.
- Callbacks : Log and stream intermediate steps of any chain.
We used the agents so far and learned about PromptTemplates (it’s inside Model I/O) but we did a little trick in Part 3 for prompting. Now we will implement it a right way (hehe). In this Part 4, we will use Chains , Memory , Agents and PromptTemplates all together.
Roll up your sleeves!
Memory
Memory is the ability to store information about past interactions. An essential component of a conversation is being able to refer to information introduced earlier in the conversation. At bare minimum, a conversational system should be able to access some window of past messages directly. You can see the doc here. I would recommend taking a look at it before we start; I will not dive into every aspect of the memory to keep this lighter.
There are several ways to add a memory in the chat. You can do it with chains, agents or you can directy add it to model too. As you know, we were using csv_agent to make a bridge for the model we use and CSV files. And we were using a big prompt to explain the data’s structure. Actually, we send this prompt in every API call (aka question), for a token-based pricing, it’s not such a clever way to do prompting like this.
What I want to create is to send that prompt before the conversation starts and keep it in the memory so I don’t not have to send it in every request that I make. And update the chat history along the way to make the model aware of the topic at the exact moment. Before getting into coding, I want to talk a bit about chains.
Chains
Chains give power to link various LLMs, and make possible to the execution of more complex tasks. For docs, go here.
You can add up chains to your LLM based on your needs, you can chain to different LLMs too. It’s basically concatenating the steps you want to make happen for your AI solution.
Implementing Memory to ChatGpt
There are couple of ways to add memory functionalty to your solution, as I mentioned. One of them is adding it in LLMChain. When you add memory within a LLMChain, the most important step is defining the input-output variables to PromptTemplate correctly because we will manually define input and outputs params. Langchains docs are pretty well explained in this blog, click here.
Other method to add memory to the LLM is adding it through an agent. But we will again use LLMChains. The flow will be like creating LLMChain with memory after creating a custom agent to use in LLMChain. To see the docs, go here.
After reading them, I first try to create LLMChain with memory and use it with a csv_agent. But csv_agent doesn’t accept LLMChains, it requires only LLM type. So I searched for a way to use csv_agent with LLMChain, but there isn’t one or I couldn’t find it. Yet we can create our own algorithm to keep the conversation in memory and to give an answer based on chat history on CSV data. The Conversation Chain will hold our chat history, and we will manage its inputs and outputs.
First of all, we will create memory and seed our previous prompt inside it like it’s happening in a converation. Later, we will add human input (question) to the chat-history. After we updated the memory, we can go to the agent for an answer over the CSV document within an input that came through the Conversation Chain. The input will contain seeded data, user’s questions, and the agent’s answers.
Please, read the code and comments.
#install to required libraries.
pip install langchain
pip install openai
pip install tabulateimport tabulate
import pandas as pd
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI
from langchain.agents import create_csv_agent
from langchain.agents.agent_types import AgentType
from langchain import PromptTemplate, LLMChain
OPENAI_API_KEY = '{your_api_key}'
#import to csv files
file_path_timesheet = '{your_file_path}'
file_path_user = '{your_file_path}'
file_path_homeoffice = '{your_file_path}'
file_path_projects = '{your_file_path}'
file_path_timeoff = '{your_file_path}'#create a ConversationBufferMemory with a pre-ready information about dataset
from langchain.memory import ConversationBufferMemory
initial_info=
"""
You are a question-answering bot over the data you queried from dataframes.
Give responses to the question on the end of the text :
df1 contains data about timesheet, which user worked on a project on that day.
Worked and spending time are representing same thing.
It's a daily record but you will not find weekend records for Turkey's timezone, GMT+03:00.
df1 columns and column descriptions as following:
date: day of record , only weekdays
user ID: unique identifier of the user
name: name of the user
email: email of the user
project_id: unique identifier of the project that user worked
hours : hours of user spend on that project
assignment_start_date : when user started to work on that project
assignment_end_date : when user ended to work on that project
df2 contains data about user detail, if you have question about user, check this out.
df2 columns and column descriptions as following:
user_id: The unique identifier of the user.
name: The name of the user.
email: The email address of the user.
employment_start_date: The date of user start the work.
birthday: The birthday of the user.
.
.
.
Below is the question
Question:
"""
memory = ConversationBufferMemory()
#seed initial data
memory.save_context(
{"input": initial_info , "history" : "" },
{"output": "okey, what is the question?"})
memory.load_memory_variables({})
final_output = ' '
output = ' ' llm = ChatOpenAI(temperature=0, model="gpt-4", openai_api_key=OPENAI_API_KEY)
## agent (chain_1)
csv_agent = create_csv_agent(
llm,
[file_path_timesheet,file_path_user,file_path_address,
file_path_homeoffice,file_path_projects,file_path_timeoff,file_path_pto],
verbose=True,
agent_type=AgentType.OPENAI_FUNCTIONS
)
# prompt_template in llm_chain (chain_2)
from langchain.chains import LLMChain
from langchain.prompts.prompt import PromptTemplate
prompt_template = """ You are a question-answering bot over the data you queryied from dataframes, answer questions only about this data.
I will give you information about where to find data on df's below.
Information of users on df2, projects information on df5,
which user worked on which project on that day information on df1.
Give responses to the question on the end of the text :
Here is the chat history:
{history}
Below is the question
Human: {input}
AI Assistant:"""
PROMPT = PromptTemplate(input_variables=["input", "history"], template=prompt_template)
#memory in prompt + llm in ConversationChain
from langchain.chains import ConversationChain
memory_chain = ConversationChain(
llm =llm,
prompt = PROMPT,
memory=memory,
verbose=True
)
# question_1
#query = 'When was the last-time Mine Kaya was working from home?'
# question_2 ((now we don't need to tell the name of the user))
#query = 'When was the last-time she had time-off?'
# question_3
query = 'how many days she have pto?'
output += memory_chain.predict(input=query)
ques = ( final_output + output + query)
final_output += csv_agent.run(ques)
It’s a bit hard to read but, you can see the answers of given questions and how agent do it’s job too.
We did it! After all these improvements, the final product was pretty good at giving answers. We tried both GPT-3.5 and GPT-4, of course, GPT-4 was certainly better, still GPT-3.5 yielded fair enough results.
I will continue to write about Generative-AI, but the project will be different in Part 2. Since the beginning, I have really enjoyed the topic. I hope you do too, see you on the next adventure.
Previous posts :
