Empowering LLMs using the LangChain Framework
In the past year, we have witnessed a massive hype surrounding Generative AI (GenAI), largely fueled by ChatGPT. LangChain has risen as the go-to framework for creating GenAI applications. LangChain allows us to create super complex applications that can search the web, access databases and leverage the reasoning capabilities of Large Language Models (LLMs) to perform autonomous tasks.
If as a developer, you’ve ever wondered:
- How do plugins in ChatGPT Plus work underneath the hood?
- How can I develop a GenAI app that executes complex tasks based on your prompts, such as initiating a Google search, interacting with SQL agents, or accessing external APIs?
Then this post is definitely for you! We will be covering why you should choose the LangChain framework, explore LangChain in numbers, go over its core modules and talk about its use cases.
Why Should You Choose LangChain?
We all know that ChatGPT has impressive general knowledge. You can ask it almost anything, and you will probably get a pretty good answer. But let’s suppose you want to know something specifically from your data – a book, a file, or a database (DB) with proprietary information. LangChain allows you to connect an LLM like ChatGPT (or Hugging Face) to your specific data. It’s not limited to pasting a text snippet to a ChatGPT prompt; it can reference an entire DB filled with your data.
LangChain also provides the ability to take action based on that data. For example, actions such as sending an email, performing a Google search, and calling any external API that you want.
Langchain in Numbers
LangChain was released in October 2022 by Harrison Chase as an open source framework and was quickly adopted by the community as the go-to framework for creating GenAI applications. It has more than 60,000 stars in Github, a lot more than its competitor LlamaIndex, and even more than traditional open source frameworks like AngularJS. It has more than 1.2 million PyPi downloads a week and 165,000 npm weekly downloads.
LangChain has a high score of 88 in the Snyk scan and doesn’t contain any vulnerabilities.
Core Modules of LangChain
The screenshot below displays several key core modules within LangChain. Chains and Agents bring together elements like LLM Wrappers, Prompt Templates, Indexes, Memory, and more, using them as fundamental building blocks.
For each of these modules, LangChain provides standard and extendable interfaces:
- LLM Wrappers — Enables connection to LLMs/Chat models.
- Prompts Templates — Enables avoiding hard code text, which is the input to the LLM. It is cleaner than just using ‘f’ strings in Python and lets you provide only the user prompts when invoking the chain or the agent.
- Indexes — Allows you to extract the relevant information from your data.
Use Cases for LangChain
Combining Langchain with LLMs has many use cases.
Some of them, such as QA over documents and Code Understanding, usually first require ingestion of data which the LLM was not trained on. This data must be ingested to a vector database and only then can the relevant data be retrieved and augmented in the prompt to the LLM.
Other use cases generally require feeding the LLM with API documentation schema, such as Interacting with APIs or Agents in order to be able to call external sources such as APIs.
The use cases are:
1) QA over Documents
Let’s consider you are using ChatGPT API, which currently has a token limit of 4,096 tokens. And you want to ask questions about a book with 200 pages.
How do we know what text should be sent from 200 pages to LLM to extract specific entities? It might be on any page/part of the document!
That is where text chunking and similarity search help to send only relevant text to LLMs to reduce tokens, thereby reducing costs.
Now that we have a better understanding of the motivation of the data ingestion phase, let’s go over the steps:
1. Load the data from any kind (images, pdf, YouTube, S3 bucket, etc.) to the document object in LangChain. You can find the updated integration list here.
2. Split documents into small chunks so that the LLM can later digest them in the prompt (without reaching the maximum context limit).
3. Chunks are inserted into a vector database after being transformed into vectors of floats, which are also called embeddings.
Cool, so now we have our proprietary data ingested in a database, but why do we need to store it in embeddings and not in text?
The following images try to answer this question.
For example, I took this great blog post, ingested it to a vector DB locally on my server, and started chatting on its data. ChatGPT was not trained on this data since it was posted on June 23, 2023.
I asked three questions :
- What is sensory memory?
- Which memory provides the ability to retain impressions of sensory information?
- Which type of memory typically only lasts for up to a few seconds?
We got answers that contain similar content. The reason for that is because
prompts are first transformed into vectors (or embeddings). Since they are semantically close in the vector space, the retrieval step from the vector database returns similar neighbor chunks, thus giving the user similar answers.
2) Chatbots
Chatbots are one of the central LLM use cases. The core features of chatbots are that they can have long-running conversations and have access to information that users want to know about.
The above diagram illustrates the logic of the LangChain chain called ChatbotConversationalRetrievalChain.
The chain contains 3 phases:
1. Use the chat history and the prompt to create a standalone question.
2. The standalone question is passed to the retriever, and only relevant documents are returned.
3. Augmenting the prompt with those documents and the standalone question-> and getting a response back.
This is done using the Retrieval Augmented Generation (RAG) method.
You can elaborate more on it here.
3) Agents
If we can create a chatbot on top of an LLM and chat over our data, why do we need agents?
The reason is that in chains in LangChain, the sequence of actions is hardcoded (in code), and in Agents, the LLM is used as a reasoning engine to determine which actions to take and in which order. This empowers the LLMs and lets us create much more complex applications.
This became feasible due to two pivotal studies (you can read them here and here) that played a key role in enhancing the capability of LLMs to execute more complex tasks.
In simple words:
Guiding the LLM to solve problems like we do as humans means breaking a complex task into small intermediate tasks. In each step, after the LLM chooses which tool to call, LangChain invokes it and returns with the response to the LLM until the problem is resolved.
We can see in the above image that the prompt was :
How many people live in Canada as of 2023? return the number raised by the power of 2 plus 5
Before the agent could resolve the task using the LLM, the Google Search tool was invoked with the following input:
current population of Canada 2023
Then, the answer to the Google Search was :38,781,291 people
The LLM did reasoning again and decided on the next tool to be invoked — a calculator with the following input — (38,781,291²) + 5
and then we get the final answer : 1503988531626686
The following screenshot tries to explain what happens behind the scenes.
There is the prompt of the user, and the LangChain agent calls the LLM to perform the reasoning, or break the task into small intermediate tasks. In this case it involves first invoking the Google Search tool and taking its output and invoking the Calculator tool, thus giving us the final resolution.
4) Interacting with APIs
Suppose you want an LLM to interact with external APIs. This is very useful for retrieving context for the LLM to utilize. More importantly, it allows us to interact with APIs using natural language.
The concept is to provide an LLM with API documentation (for example, open API spec) and then ask questions in natural language. The LLM then concludes the REST API call and performs the call, answering the user.
You can learn more about interacting with APIs here.
5) Tagging
Tagging means labeling a document with classes such as:
- sentiment
- language
- style (formal, informal, etc.)
- covered topics
- political tendency
Tagging has many use cases, such as analyzing customer cases.
To demonstrate the behavior in the screenshot above, I used the LangChain tagging chain that extracts information from a passage based on a schema.
I ran the exact scenario from the screenshot above, and this is what I received as an answer:
You can learn more about tagging here.
6) Summarization
Suppose you have a set of documents (PDFs, Notion pages, customer questions, etc), and you want to summarize the content. LLMs are a great tool for this, given their proficiency in understanding and synthesizing text.
As an example, I took this great blog post and used the LangChain load summarizing chain, which is provided off the shelf, and asked it to summarize the content of this blog.
Here is the answer:
Nice right?
You can learn more about summarization here.
7) SQL Using Natural Language
Enterprise data is often stored in SQL databases and LLMs make it possible to interact with SQL databases using natural language. LangChain offers SQL Chains and Agents to build and run SQL queries based on natural language prompts.
These are compatible with any SQL dialect supported by SQLAlchemy (e.g., MySQL, PostgreSQL, Oracle SQL, Databricks, SQLite).
They enable use cases such as:
- Generating queries that run based on natural language questions
- Creating chatbots that can answer questions based on database data
- Building custom dashboards based on insights a user wants to analyze
You can learn more about SQL using natural language here.
The Generative AI Revolution
We are witnesses to a GenAI revolution that will transform many aspects of our lives. While the industrial revolution was primarily about advancements in tools and processes, the AI revolution is focused on the replacement of human skills. Using the LangChain framework, we can embrace this change and create really powerful applications powered by LLMs such as image generation, chatbots, sentiment analysis and code generation and many more.
