LangChain components. Why and How?

12 min readJul 22, 2024

Imagine having a virtual assistant that can handle complex tasks such as data analysis, customer service, and content generation, all through simple natural language commands — just like JARVIS or EDITH (for fellow Iron Man fans). This is the potential of AI combined with LangChain, and today, I’ll show you how to unlock this power.

This article isn’t just about explaining what LangChain is and its components — you can find that information elsewhere on the internet. Instead, it will delve into:

Why these components are used?
Why they are important?

While an LLM (Large Language Model) typically involves a single model that performs tasks such as generating text based on a topic and language, LangChain integrates multiple models to create a more complex workflow. For example:

LLM : Generates a paragraph based on a topic and language.
LangChain :
Node 01: Generates a paragraph based on a topic and language.
Node 02: Translates the paragraph to English.
Node 03: Reviews the translated paragraph.

In LangChain, each node can use different models, and the output of one node is provided as input to the next.

LangChain consists of several components. Let’s explore each one and understand how they interconnect.

1. Models :

A model is essentially a large neural network trained to understand and generate natural language responses based on the data it has been trained on. LangChain offers a standardized interface to connect various models, including LLMs, chat models (like ChatGPT), and other text embedding models. Whether the model is online, offline, paid, or free, LangChain supports it.

For example, the following code demonstrates how to access OpenAI chat models:

from langchain.llms import OpenAI

# Initialize the language model
llm = OpenAI(model='text-davinci-003')

# Define a prompt for translation
prompt = "Translate the following text to French: 'Artificial intelligence is transforming the world.'"

# Generate translated text
response = llm(prompt)
print(response)

2. Prompt :

To understand what a prompt is, let’s ask ChatGPT directly:

Detailed Scenario-Based Prompt with Context

Notice that even though we’re asking similar questions with slightly different instructions, the core concept of a prompt remains consistent. It’s the input used to drive the model’s response.

The following code demonstrates how to call the ChatGPT model without using any prompt:

from langchain.llms import OpenAI
import os
os.environ['OPENAI_API_KEY'] = '<your-openai-api-key>'
llm = OpenAI(model_name="text-curie-001", temperature=0.5, n=1, max_tokens=100)
llm("Tell me a something about the universe.")

In contrast, the next example shows how to call the ChatGPT model using PromptTemplate (a library by OpenAI). This approach allows you to add context before making your query:

instructional_template = "Write a brief summary of the following text: {text}"
instructional_prompt = PromptTemplate.from_template(instructional_template)
instructional_input = instructional_prompt(text="Artificial Intelligence is transforming various industries by enabling new capabilities and efficiencies.")
instructional_response = llm(instructional_input)
print(instructional_response)

Prompts can influence the style and tone of the response. If you ask, “Explain quantum physics to a 5-year-old,” the model will simplify the explanation. If you ask, “Give a technical overview of quantum physics,” the response will be more complex.

A well-crafted prompt provides clear direction. Prompts can shape the clarity and detail of the response. Prompt sets the context for the answer. By including specific details or constraints, you can get responses that are more aligned with what you need. Prompt is like a guide that steers the model’s responses in the direction you want. Changing the prompt can result in significantly different answers because it alters the focus and detail of the information the model generates.

The ability to optimize prompts not only demonstrates a deep understanding of AI capabilities but also showcases a strategic approach to problem-solving, making it a highly sought-after skill that can open doors to lucrative positions and impactful projects(It is a million dollar job).

There are many types of prompts : 1. Instructional Prompt 2. Conversational Prompt 3. Role-Based Prompt 4. Contextual Prompt and many more.

Examples for each type of prompt, along with corresponding code, are provided in the Colab notebook below.

3. Memory :

Let’s first take example of one of the widely used text based model ChatGPT. Suppose I am asking a simple question to ChatGPT, like given below.

Now, if you ask another question in the same chat.

See the example, Even though I haven’t specified the topic, ChatGPT can still provide an accurate answer. So how does it does this? It recalls the earlier question about the memory in LangChain and infers that “types” refers to memory types in langChain.

So, the answer is Memory. ChatGPT uses memory to recall previous conversations and provide relevant responses to recent questions based on that context.

But how does it achieve this? The simplest explanation is through the use of Prompts. For example, it utilizes different types of prompts to maintain and leverage context:

{'input': 'Good morning AI!',
 'history': '',
 'response': " Good morning! It's a beautiful day today, isn't it? How can I help you?"}

This is a simple example of how memory is utilized: essentially, it involves adding some extra context to the prompt that we pass to the LLM while asking the most recent question. This context includes information from the chat history.

Chat history is maintained as follows: each time the user asks a new question and receives an answer, both the question and the answer are added to the chat history. Here’s an illustration of how chat history might look:

User: "What is the capital of France?"
AI: "The capital of France is Paris."
User: "What is the population of that city?"
AI: "The population of Paris is approximately 2.1 million."

However, using chat history increases the number of tokens consumed. This can be problematic for several reasons:

Cost: LLMs like ChatGPT often charge based on the number of tokens processed. More tokens mean higher costs.
Token Limit: LLMs have a maximum token limit for each input. If the chat history becomes too long, it can exceed this limit, requiring truncation or omission of earlier context.
Efficiency: Managing large amounts of chat history can slow down processing and reduce the efficiency of generating responses.

Therefore, while maintaining context is crucial for accurate responses, balancing it with token usage is important to manage costs and performance.

To manage token usage while preserving context, various types of memory can be used:
1. Conversation Buffer Memory
2. Conversation Summary Memory
3. Conversation Topic Memory
4. Conversation Buffer Window Memory
5. Conversation Summary Buffer Memory
6. Vector Store-Backed Memory
In-depth examples of each type are provided in the Colab notebook below.

4. Chains :

A chain is a structured sequence of operations where each operation (or link) performs a specific task, and the output of one operation is passed as input to the next. Chains link together various steps to achieve a particular goal.

In LangChain, chains involve multiple components, such as language models, prompts, and tools, organized in a sequence. Each step in the chain builds on the previous one, making the overall process more organized and efficient.

Essentially, chains allow you to create complex AI applications by combining multiple simpler operations in a logical sequence, following example shows the code for creating chain using langChain.

from langchain.llms import OpenAI
from langchain.chains import SequentialChain
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

# Initialize the language models
llm1 = OpenAI(model='text-davinci-003', temperature=0.8)
llm2 = OpenAI(model='text-davinci-003', temperature=0.0)
llm3 = OpenAI(model='text-curie-001', temperature=0.5)

# Define the prompts
prompt1 = PromptTemplate.from_template("Write a short paragraph about {topic} in {language}.")
prompt2 = PromptTemplate.from_template("Translate the following paragraph into English: {paragraph}")
prompt3 = PromptTemplate.from_template("Review the writing style of this paragraph: {english_paragraph}")

# Create the chains
chain1 = LLMChain(llm=llm1, prompt=prompt1, output_key="paragraph")
chain2 = LLMChain(llm=llm2, prompt=prompt2, output_key="english_paragraph")
chain3 = LLMChain(llm=llm3, prompt=prompt3, output_key="review")

# Combine the chains into a sequential chain
combined_chain = SequentialChain(chains=[chain1, chain2, chain3],
                                 input_variables=['topic', 'language'],
                                 output_variables=['paragraph', 'english_paragraph', 'review'],
                                 verbose=True)

# Run the combined chain
result = combined_chain({'topic': 'Napoleon Bonaparte', 'language': 'Spanish'})
print(result)

In the example provided, three models are used, each with an assigned prompt template. The chain is created using chains=[chain1, chain2, chain3], where the combined_chain function manages the flow of data.

Here’s how it works:

Chain 1: Takes the initial input and generates a paragraph as output.
Chain 2: Receives the output from Chain 1 (the paragraph) and translates it to English.
Chain 3: Takes the English translation from Chain 2 and performs a review.

The output of each chain is passed as input to the next chain in the sequence. The final output, which is the review, is returned by the third chain.

Chains provide a clear and organized way to define and execute a sequence of steps. Each component in the chain builds on the output of the previous one, ensuring a systematic approach to complex tasks.

5. Indexes :

Suppose you have data about your hotel and want to create a chatbot to interact with users. Since LLM models like ChatGPT do not have specific knowledge about your hotel data, how you gonna create chatBot? So mainly two methods you gonna have in your mind

Create your own model on your own data: This involves training a model specifically on your hotel's data to create a chatbot tailored to your needs. However, this approach can be time-consuming and resource-intensive.
Use a Pre-Trained Model with High Accuracy: Leveraging a pre-trained model like GPT, which already has high accuracy and general language understanding, can be more efficient.

You would likely choose the second method but now the question is how you gonna do it? Answer is Retrieval Augmented Generation(RAG) or in simple words Prompt Engineering.

How prompt engineering?

Extract and Split Data: Start by extracting text from your PDF or other documents containing hotel data. Split this text into manageable chunks or sections.
Query Processing: When a user asks a question related to the hotel, identify and retrieve the most relevant chunk of text from the extracted data.
Contextual Prompt: Use the retrieved text as context and incorporate it into your prompt to the language model. This allows the model to generate a response based on both the pre-trained knowledge and the specific hotel data.

prompt = "You are a personalized chatBot for hotel.
          Here is some information about our hotel: '{relevant_text}'. 
          Based on this, 
          Try to answer following user query as best as you can: '{user_query}'"

Indexes are structures that help organize and retrieve information efficiently from large database. They are used to store and manage data in a way that allows for quick and accurate access, particularly when dealing with large datasets or documents.

Indexes are crucial for tasks like information retrieval, search, and question answering, as they enable the system to find relevant data without having to scan through all available information.

We’re just scratching the surface here. Get ready for an in-depth exploration of Retrieval-Augmented Generation (RAG) in our next blog. You won’t want to miss it — stay tuned for the big reveal!

6. Agents :

Now, let’s introduce our hero — the future of AI: Agents.

Why do we need agents? To understand this, let’s start with a simple example. Imagine asking basic math questions to our LLM model.

Recent ChatGPT models uses tools to answer the queries.

In the example below, you’ll see a little code icon beside the text. Click on that icon, and it will pop up the following window.

When you click on the code icon, you’ll see that a Python code snippet is generated for your question. This code is then executed, and the results are returned to answer your query. What you’re observing is an Agent — specifically, a MathAgent.

LLMs are powerful text models trained on vast amounts of data, but they can’t cover every possible scenario, especially in fields like mathematics where the possibilities are endless. Instead of training the model on every conceivable math question, ChatGPT generates Python code for the math problem, executes it, and returns the result. This approach reduces the reliance on the model itself to handle complex tasks.

Chains involve a fixed sequence of actions that are hardcoded into the system.
Agents, on the other hand, use the language model as a reasoning engine to determine which actions to take and in what order. This dynamic approach allows agents to adapt and make decisions based on the context and requirements of each specific task.

Action Agents are agents that decide the action to be taken & then execute the action repeating this until they achieve their objective.

In the ChatGPT interface, what you’re seeing is a component called PythonReplTool. Now you gonna ask what is a tool now?

Imagine a scenario where the CEO represents the LLM Chain. The CEO assigns tasks to various managers, which correspond to the Agents. To complete these tasks, each manager (Agent) breaks down the tasks into smaller, manageable pieces and assigns these to their employees, which are the Tools.

Just as a single manager can have multiple employees handling different tasks, an agent can use various tools to perform specific functions.

Putting it all together:

LLM Chains or LangChain can feature multiple Agents (experts in their own domains).
Each Agent can deploy multiple Tools (designed to handle particular tasks).

This architecture of agents and tools allows us to harness the true power of LLMs, enabling complex, adaptive, and highly specialized capabilities.

So, how do you create an agent using code? Let’s look at a common example of how to build a simple ReAct (Reason + Act) agent. This example covers various components of LangChain: memory, prompts, LLMs, tools, and agents.

# Initialize LLM
llm = OpenAI(api_key='your-api-key')

# Define a prompt template
prompt_template = PromptTemplate(
    input_variables=['context', 'query'],
    template="Given the context: {context}, answer the query: {query}"
)

# Set up memory
memory = ConversationBufferMemory()

# Configure tools
tool = PythonReplTool()

# Create an agent
agent = Agent(
    llm=llm,
    prompt_template=prompt_template,
    memory=memory,
    tools=[tool]
)

# Define a function to interact with the agent
def interact_with_agent(user_query):
    context = memory.get_recent_context()  # Retrieve recent context from memory
    response = agent.run(context=context, query=user_query)  # Run the agent with context and query
    return response

# Example usage
user_query = "What is the square root of 16?"
result = interact_with_agent(user_query)
print(result)

We’ll dive deep into the world of agents in our upcoming blogs. Stay tuned for more exciting insights! :)

Imagine if your LLMs had a superhero upgrade: that’s what agents are all about! With their superpower of reasoning and action, agents turn our LLMs from mere text-generating machines into decision-making dynamos. They orchestrate tasks with the finesse of a maestro and the efficiency of a Swiss watch. So, buckle up and get ready — agents are here to unlock the full potential of LLMs and make your AI experience a bit more “super”!

To wrap things up, let’s test our understanding of LangChain with a few questions. These are some of the questions that popped into my mind as I explored LangChain. Ready to challenge yourself?

1. What are the main components of LangChain, and how do they interact to build a conversational AI system?
2. Provide an example scenario where one type of memory is preferable over the other.
3. Explain how vector stores like FAISS are used in LangChain to manage document retrieval. What are the benefits of using a vector store?
4. How do agents orchestrate the interaction between different components in a workflow?
5. What are the key considerations when designing an agent for a specific task in LangChain?
6. How would you approach debugging and optimizing a LangChain-based application that is not performing as expected?
7. What metrics would you use to evaluate the performance of a LangChain application, and why?
8. Long conversations can lead to context loss or confusion. How would you manage and maintain context effectively in a LangChain application designed for long conversations?
9. Do langchain really require GPU as we are using prebuild models?

This article isn’t another copy pasted version of any other blog out there, it’s a reflection of what I’ve learned on my AI journey over the past two months. As this is my very first blog, there may be some mistakes, feel free to point them out below in comments and help me improve.

References :

Colab NoteBook : https://colab.research.google.com/drive/1DeG5nYn1td5Bbhgk5vKaoCR0eifFRXJZ?usp=sharing

LangChain docs : https://python.langchain.com/v0.2/docs/introduction/

Video Tutorial I referred : https://www.pinecone.io/learn/series/langchain/langchain-intro/