Using LangChain for LLM application development: a simple data analysis tool

Published in

Indicium Engineering

7 min readJun 13, 2024

Unlocking the potential of LangChain is game-changing: this framework is revolutionizing the way we develop applications with large language models (LLMs).

We’ll cover essential components such as models, prompts, and parsers, and then dive into more advanced topics like chains, memory, and evaluating model outputs.

This post will be particularly useful for those looking to build effective language model applications.

Models

A model in the context of language models refers to the specific configuration of a neural network trained on a large corpus of text data to understand and generate human-like text.

For this tutorial, we are using OpenAI’s GPT-3.5 models, which can be controlled using parameters like temperature to influence the randomness and creativity of the generated responses.

Why models matter

The choice of model impacts the quality, coherence, and relevance of the generated text. For instance, a lower temperature results in more deterministic and conservative outputs, which is useful for tasks requiring precise and accurate answers.

Conversely, a higher temperature increases randomness, fostering creativity and exploration in the generated responses. Below is an example of initializing the the language model.

import os
from dotenv import load_dotenv, find_dotenv
from langchain_openai import ChatOpenAI

# Load environment variables from .env file
_ = load_dotenv(find_dotenv())
openai.api_key = os.getenv('OPENAI_API_KEY')

# Initialize the OpenAI language model
llm = ChatOpenAI(temperature=0.7, model="gpt-3.5-turbo")
# Or some other model you plan on using

Prompts

Prompts are the input queries or instructions given to a language model to generate a response. Crafting effective prompts is essential for getting the desired output from the model.

Prompts can range from simple questions to complex instructions or templates with placeholders for dynamic content.

Effective prompting

Effective prompting guides the model to produce accurate and relevant responses.

A well-structured prompt might include context, specific instructions, and desired formats. This structure helps the model understand the task better and generate more useful outputs.

LangChain and prompts

LangChain simplifies creating and using prompts by providing utilities for prompt templates and chaining multiple prompts together.

This approach ensures that our prompts are consistent, easily maintainable, and adaptable to different contexts or tasks.

LangChain’s ChatPromptTemplate can be used to create structured and reusable prompts. There are a couple different ways to structure these prompts, as shown below.

from langchain.prompts import ChatPromptTemplate

# Prompt for selecting analysis techniques
intent_and_analysis_prompt = ChatPromptTemplate.from_template(
   """ The user wants to analyze the data. Here is their explanation: {user_explanation}
Based on this explanation, select all the relevant analysis techniques from the following options: {analysis_options}.
Provide a list of the selected analysis techniques in the format of a comma-separated list.
   """
)

# Prompt for identifying relevant columns
columns_string = """ User wants to analyze the data with the following 
columns: {user_columns}.
Given the user's explanation: {user_explanation}, identify the most relevant columns for the analysis.
Provide a list of the selected columns in the format of a comma-separated list.
   """

columns_prompt = ChatPromptTemplate.from_template(columns_string)

Parsers

After the LLM is given the prompt, it will provide an output. Parsers in LangChain are used to interpret the output generated by the language model and extract necessary information.

This is particularly useful for structured data or when the output needs further processing.

Why parsers are Essential

Parsers help by extracting relevant information, converting it into structured formats (e.g., JSON), and ensuring that the output meets specific criteria required by subsequent processing steps or applications.

Chains: simple, sequential, and router

Chains in LangChain are a powerful feature that allows you to combine multiple prompts and parsers into a cohesive workflow.

These chains enable the creation of more complex and versatile applications by structuring how tasks are executed and how data flows through the system.

Simple chains

Simple chains are ideal for linear processes where each step’s output directly informs the next step.

In a simple chain, each task is performed in a straightforward sequence, with the result of one task becoming the input for the next. This type of chain is useful for straightforward workflows where the progression from one step to the next is direct and uncomplicated.

Sequential chains

Sequential chains involve a series of steps that must be executed in a specific order. These steps build on each other, creating a chain that must be completed from start to finish.

Each step’s output becomes the input for the subsequent step, ensuring that the entire sequence is followed meticulously. Sequential chains are ideal for complex workflows where each task depends on the completion of the previous one, ensuring that the entire process flows logically and coherently from beginning to end.

A sequential chain is shown below:

# Prompt for understanding user's intent and selecting relevant analysis techniques
intent_and_analysis_prompt = ChatPromptTemplate.from_template(
   """
   The user wants to analyze the data. Here is their explanation: {user_explanation}
   Based on this explanation, select all the relevant analysis techniques from the following options:
   {analysis_options}.
   Provide a list of the selected analysis techniques in the format of a comma-separated list.
   """
)
# first chain
intent_and_analysis_chain = LLMChain(llm=llm, prompt=intent_and_analysis_prompt, output_key="selected_analysis_types")

# Prompt for identifying relevant columns
columns_prompt = ChatPromptTemplate.from_template(
   """
   User wants to analyze the data with the following columns: {user_columns}.
   Given the user's explanation: {user_explanation}, identify the most relevant columns for the analysis.
   Provide a list of the selected columns in the format of a comma-separated list.
   """
)
# second chain
columns_chain = LLMChain(llm=llm, prompt=columns_prompt, output_key="selected_columns")

# Overall sequential chain
overall_chain = SequentialChain(
   chains=[intent_and_analysis_chain, columns_chain],
   input_variables=["user_explanation", "user_columns", "df", "analysis_options"],
   output_variables=["selected_analysis_types", "selected_columns"],
   verbose=True
)

Router chains

Router chains involve conditional branching, where the workflow path can change based on certain criteria.

In a router chain, the system can dynamically decide which path to take based on the input or intermediate results. This allows for more flexible and adaptable workflows that can handle various types of queries or tasks. Router chains are particularly useful for applications requiring dynamic decision-making and can manage diverse tasks by routing them to the appropriate processing chain based on specific conditions or criteria.

Memory

In LangChain, memory modules are essential for maintaining context across interactions with a language model. This enables the model to provide more coherent and contextually appropriate responses.

We’ll explore two types of memory: ConversationBufferMemory and ConversationSummaryMemory. Although these techniques are not explored in the supporting code, I have provided sample code below for reference.

ConversationBufferMemory

ConversationBufferMemory stores all past interactions in a buffer, allowing the model to reference previous exchanges to maintain context throughout a conversation.

This is particularly useful for applications where continuous context is crucial, such as customer service chatbots or interactive story generation.

In the example below, we initialize a language model, create a memory buffer to store conversation history, set up a conversation chain, and conduct a conversation where the model maintains context by referencing the memory buffer.

from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory

llm = ChatOpenAI(temperature=0.0, model="gpt-3.5-turbo")
memory = ConversationBufferMemory()
conversation = ConversationChain(llm=llm, memory=memory, verbose=True)

conversation.predict(input="Hi, my name is Andrew")
conversation.predict(input="What is 1+1?")
conversation.predict(input="What is my name?")
print(memory.buffer)

ConversationSummaryMemory

ConversationSummaryMemory summarizes past interactions, condensing the information into a more manageable format while preserving the essential context.

This is useful for applications where a concise summary of the conversation is needed rather than the full history.

In the example below, we create a summary memory buffer to store and summarize conversation history. After saving several context pairs, we load and print the summarized memory, and continue the conversation using the summarized context to maintain coherence.

from langchain.memory import ConversationSummaryBufferMemory

schedule = ("There is a meeting at 8am with your product team. "
 "You will need your powerpoint presentation prepared. "
 "9am-12pm have time to work on your LangChain project. "
 "At Noon, lunch at the Italian restaurant with a customer.")

llm = ChatOpenAI(temperature=0.0, model="gpt-3.5-turbo")
memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=100)

memory.save_context({"input": "Hello"}, {"output": "What's up"})
memory.save_context({"input": "Not much, just hanging"}, {"output": "Cool"})
memory.save_context({"input": "What is on the schedule today?"}, {"output": f"{schedule}"})
print(memory.load_memory_variables({}))

conversation = ConversationChain(llm=llm, memory=memory, verbose=True)
conversation.predict(input="What would be a good demo to show?")

Evaluating the output

Evaluation is a critical step in developing applications with language models. It involves assessing the quality, relevance, and accuracy of the outputs generated by the model, ensuring it meets the application’s requirements.

Quality metrics, such as precision, recall, and F1 score, provide quantitative measures of the model’s performance. Precision measures the proportion of relevant instances among the retrieved instances, recall measures the proportion of relevant instances retrieved, and the F1 score balances both by considering their harmonic mean.

Human evaluation plays a crucial role in assessing the quality and relevance of model outputs. Human annotators review the outputs to provide qualitative insights that automated metrics might miss, such as tone, style, and contextual appropriateness.

For tasks like translation or summarization, automated metrics such as BLEU and ROUGE scores offer standardized ways to measure output quality against reference texts. BLEU evaluates machine-translated text against reference translations, while ROUGE measures the overlap of n-grams, word sequences, and word pairs between the generated summary and reference summaries.

Combining automated metrics with human evaluation provides a comprehensive assessment of the model’s performance.

This dual approach ensures that both quantitative and qualitative aspects are evaluated, guiding improvements and refinements in the application. By leveraging these methods, developers can enhance the effectiveness and reliability of their language model applications.

To Review…

By using LangChain, we can streamline the process of interacting with language models, creating dynamic and context-aware prompts, and parsing the results effectively.

This not only improves the development workflow but also enhances the quality and consistency of the outputs generated by language models.

Supporting code on Github

You can find the supporting complete code in the GitHub repository. This demonstrates the processes outlined above for creating a simple LLM project with Langchain (not including implementing memory). This includes:

The sequential_chain.py script.
A requirements.txt file
A README.md file to guide you through this process