How to Build Your Own Chatbot with LangChain and OpenAI

In this tutorial, we will be creating a chatbot built for a specific use-case using LangChain and OpenAI. If you’re just getting on board the LLM hype train and don’t know much about it yet, you’re in the right place. This tutorial is easy to understand and follow, even for beginners.

This is the first part of a three-part series. We’ll start with creating a chatbot for a particular use-case in this article, then move on to building an application in the second tutorial, and finally, learn how to link it up with external data.

Image from author

Imagine you are just starting out with an ice-cream business and want to learn everything about ice-creams (your ideal customers, unique flavor combinations, easy ice-cream recipes etc.). Let’s call our ice-cream assistant Scoopsie. Our starting point is a simple use-case, and we’ll develop our chatbot using LangChain and OpenAI’s text completion model. If you haven’t heard of it, LangChain is a framework that makes it easier to build applications with large language models by offering sufficient abstraction.

Environment Setup

For a clean technical implementation, I prefer setting up a new Conda environment for each project. If you are unfamiliar with Conda, you can also set up a virtual environment in the IDE of your choice. In the next steps I will be detailing how to create a new Conda environment, activating it, and installing all necessary libraries in the environment.

conda create --name chatbot_langchain python=3.10

To activate your environment, simply type:

conda activate chatbot_langchain

Now that you have created a new Conda environment specific to this project, go ahead and install all the required dependencies. You can do this by using the following command in the same terminal after activating your environment.

pip install -r requirements.txt

We are now ready to start working on the code for the chatbot itself.

Step-by-Step Implementation

Step 1:

First of all, we will create a simple prompt to query our large language model. In this example, we will be using a text completion model from OpenAI, specifically the gpt-3.5-turbo-instruct. For this project, we are building a chatbot that will help the users of the chatbot answer any ice-cream related queries. The user can ask the chatbot any question related to ice-cream (ice-cream flavor combinations, recipes, even ice-cream jokes). However, we would like for our chatbot to not answer queries unrelated to ice-creams. We can encode all of these instructions into our prompt.

To keep the codebase modular, we will create a new python script prompts.py that houses all of our prompts. You can then pass the prompt to a prompt template from LangChain, and import that to your main script. The PromptTemplate can be thought of as a wrapper around prompts, where you can specify the input variable, as well as any other key information.

from langchain.prompts import PromptTemplate

Below is the prompt that we will pass along with the user query. This is a simple instruction that tells our ice-cream assistant chatbot what it is expected to do.

ice_cream_assistant_template = """
You are an ice cream assistant chatbot named "Scoopsie". Your expertise is 
exclusively in providing information and advice about anything related to ice creams. This includes flavor combinations, ice cream recipes, and general 
ice cream-related queries. You do not provide information outside of this 
scope. If a question is not about ice cream, respond with, "I specialize only in ice cream related queries." 
Question: {question} 
Answer:"""

ice_cream_assistant_prompt_template = PromptTemplate(
    input_variables=["question"],
    template=ice_cream_assistant_template
)

I have named our ice-cream assistant Scoopsie. You can name yours anything or nothing, depending on preference. Custom chatbots are typically given a name, and this also helps us validate our prompt when we query the model. When you query the chatbot asking it for its name, it should say it is Scoopsie. Similarly, if you ask it something unrelated to ice-creams, it should say it doesn’t know the answer.

Step 2:

In this tutorial, we will be using the gpt 3.5 model from OpenAI. You can sign up at OpenAI and obtain your own key to start making calls to the gpt model. Once you have the key, create a .env file in your repository and store the OpenAI key.

OPENAI_API_KEY="your_unique_key_goes_here"

For this tutorial, we will be using the python-dotenv library to access this key elsewhere in our codebase. Python-dotenv reads key-value pairs from a .env file and can set them as environment variables. The library would already be installed in your conda or virtualenv from the environment setup. In your main python script, you would need to call the following function from the library:

from dotenv import load_dotenv

load_dotenv()

Step 3:

The next set of codes will be assembled in a new python script called chatbot.py. This is the main script that we will be running to interact with our model. However, for ease of understanding, I have broken it into smaller components and I will explain these in the next few steps. First of all, we will create an instance of the OpenAI class, configured to utilize a specific model with a specific temperature value.

from langchain_openai import OpenAI

llm = OpenAI(model='gpt-3.5-turbo-instruct',
             temperature=0.7)

After the updates on January 4, 2024, OpenAI deprecated a lot of its models and replaced them with new ones. Make sure to use a functional model even if you are using an older version of the LangChain library. During instantiation, we have defined the model name, and set a value for temperature. In large language models like OpenAI’s, “temperature” controls text generation randomness. Low temperatures (0 to 0.5) yield predictable, deterministic text, while high temperatures (above 1.0) increase creativity and unpredictability. A value between 0.5 and 1.0 is a good balance between randomness and predictability.

Step 4:

Next, we need to create a LLM chain. In LangChain, LLM chains represent a higher-level abstraction for interacting with language models. While we can use the direct LLM interface in our simple chatbot, the LLMChain interface wraps an LLM to add additional functionality. With chains, we can have prompt formatting and input/output parsing, and they are used extensively by higher level LangChain tools. For our simple chatbot, we will use the LLMChain, and pass it the model object we created, alongwith our ice-cream assistant prompt template.

from langchain.chains import LLMChain

llm_chain = LLMChain(llm=llm, prompt=ice_cream_assistant_prompt_template)

Step 5:

Our simple chatbot’s code is mostly complete. We just need to create an entry point for our script aka the main function. We also need to be able to query our model. This is done using the invoke function with the llm_chain object. Putting together everything we have done so far in our chatbot.py script, you will have something that looks like this:

from langchain_openai import OpenAI
from langchain.chains import LLMChain
from prompts import ice_cream_assistant_prompt_template

from dotenv import load_dotenv

load_dotenv()

llm = OpenAI(model='gpt-3.5-turbo-instruct',
             temperature=0)

llm_chain = LLMChain(llm=llm, prompt=ice_cream_assistant_prompt_template)


def query_llm(question):
    print(llm_chain.invoke({'question': question})['text'])


if __name__ == '__main__':
    query_llm("Who are you?")

Testing and Validation

Let’s now validate the performance of our bot. We have not integrated any memory into our bot as of now, so each query would have to be its own function call. Specifically when testing any llm application/bot, I like to start with a few basic questions.

if __name__ == '__main__':
    question = "Who are you?"
    query_llm(question)

Chatbot’s response when asked “Who are you?”

if __name__ == '__main__':
    question = "What is 2+2?"
    query_llm(question)

Chatbot’s response when asked “What is 2+2?”

question = "I need chocolate ice-cream recipe"
print(llm_chain.invoke({'question': question})['text'])

Chatbot’s response when asked about an ice-cream recipe

Next Steps

Our simple use-case specific chatbot is now ready. In the next tutorial, we will be focusing on integrating a history component to our bot so that our application resembles the flow of a conversation, instead of standalone inquiries, as well as creating an application.

You can find the code on my GitHub. I have created a GitHub checkpoint to make it easier to follow the code along with each tutorial.

You can follow along as I share working demos, explanations and cool side projects on things in the AI space. Come say hi on LinkedIn! 👋