LangChain: An Introductory Guide (Part 1)

LangChain is an an open-source framework designed to take your AI applications to a new level of sophistication.

Introduction

LangChain is rapidly carving its niche in the AI arena, and for a good reason. It serves as a bridge connecting large language models (LLMs), such as GPT-2 (if you want to train your custom bot) or GPT 3.5 or Dolly, to external data sources. This capability not only broadens the AI’s knowledge spectrum but also enables access to up-to-date information, surpassing the traditional limitation of language models only knowing as much as their latest training data, which for most models is only till 2021.

Langchain Overview (ref)

Getting Started

Installation is as simple as running the pip install command:

!pip install langchain

Next, secure your OpenAI and Pinecone API keys from their respective websites, and initialize them as environment variables.

Using GPT-3.5 with LangChain

LangChain provides an intuitive wrapper for interacting with GPT-3.5. Below is an example of how to invoke a conversation with this model:

from langchain.schema import AIMessage, HumanMessage, SystemMessage
from langchain.chat_models import ChatOpenAI

conversational_ai = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.3)
msg = [
    SystemMessage(content="You are an amazing music recommender"),
    HumanMessage(content="Recommend me a rock song from the 80s")
]
reply = conversational_ai(msg)
print(reply.content)

Prompt Templates

Creating reusable prompts can streamline your LangChain experience. The PromptTemplate class in LangChain lets you define custom templates to avoid redundant hardcoding:

from langchain import PromptTemplate

book_recommender_template = """
You're an AI who knows all about books.
Recommend 3 good books related to the topic of {theme}.
"""

reusable_prompt = PromptTemplate(
    input_variables=["theme"],
    template=book_recommender_template,
)

Now, you can obtain great book recommendations about any topic by swapping the “theme” variable in the PromptTemplate.

Chains: Adding a New Dimension

Chains enable you to integrate your PromptTemplates and LLMs, creating composite functions to run complex operations in a streamlined manner. To illustrate, we can utilize the wrappers and PromptTemplates to run a chain with a single command:

from langchain.chains import LLMChain
chain_example = LLMChain(llm=conversational_ai, prompt=reusable_prompt)
print(chain_example.run("Horror in 1800's"))

But it doesn’t end here. LangChain allows you to string these chains together, creating larger and more complex operations. This interplay between chains and templates is where LangChain truly shines. Here is an example:

from langchain.chains import SimpleSequentialChain

explanation_prompt = PromptTemplate(
    input_variables=["theme"],
    template="""
Give me 3 additional details on the the books you suggested on {theme}
1. The Author
2. number of copies sold
3. Brief introduction to the story
"""
)

second_chain = LLMChain(llm=conversational_ai, prompt=explanation_prompt)
overall_sequence = SimpleSequentialChain(chains=[chain_example, second_chain], verbose=True)

chain_ans = overall_sequence.run("Horror in 1800's")
print(chain_ans)

We’ve just begun to scratch the surface of LangChain’s potential. In the subsequent article, we’ll explore more advanced concepts such as Vector Stores and Agents to further leverage the framework’s capabilities.

If you’re interested in delving deeper on the concepts covered, I urge you to try running my notebook and get a more hands-on feel of the framework.

happy coding!