Analysis of HayStack 2.0 generators: Is it better than LangChain?
In the dynamic landscape of Natural Language Processing (NLP), HayStack is another open-source framework tailored for building robust applications powered by Large Language Models (LLMs). It claims of offering a versatile suite of features, Haystack facilitates the creation of Retrieval-Augmented Generative (RAG) pipelines, semantic document search systems, and question answering engines, catering to a diverse range of search use cases. With the release of Haystack 2.0, the framework undergoes significant enhancements, refining Document Stores and Pipelines to optimize scalability and flexibility.
Haystack, offers a glimpse into the potential of text generation through its Generator component. Generators within Haystack serve as the linchpin for generating text outputs based on user prompts, tailored to specific LLMs. Here, we explore the capabilities of Haystack’s Generators, specifically focusing on the AzureOpenAIChatGenerator and AmazonBedrockChatGenerator, and their integration within the Haystack pipeline.
AzureOpenAIChatGenerator
AzureOpenAIChatGenerator component supports OpenAI models deployed through Azure services.
- Package installation is required before using any component of Haystack.
pip install haystack-ai- Importing the needed dependencies for this component
from haystack.components.builders import DynamicChatPromptBuilder
from haystack.components.generators.chat import AzureOpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack import Pipeline
from haystack.utils import Secret- You will need your azure OpenAI endpoint, key and version for using the component here.
prompt_builder = DynamicChatPromptBuilder()
llm = AzureOpenAIChatGenerator(azure_endpoint="",
api_key=Secret.from_token(""),
azure_deployment="")
pipe = Pipeline()
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)
pipe.connect("prompt_builder.prompt", "llm.messages")- And now we run the pipeline with our question.
variable = "Microsoft Azure"
messages = [ChatMessage.from_system("Provide summary in bullet points for any topic asked by user. Make sure to include examples as well"),
ChatMessage.from_user("Tell me about {{variable}}")]
pipe.run(data={"prompt_builder": {"template_variables":{"variable": variable}, "prompt_source": messages}})
AmazonBedrockChatGenerator
AmazonBedrockChatGenerator enables chat completion using chat models from Anthropic, Cohere, and Meta Llama 2 with a single component. We have used Anthropic for our tests.
- As usual, it starts with installing the required package for this specific component.
pip install amazon-bedrock-haystack- And then you import the dependencies.
from haystack import Pipeline
from haystack.components.builders import DynamicChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator- You will need the model name for using the Bedrock component here.
pipe = Pipeline()
pipe.add_component("prompt_builder", DynamicChatPromptBuilder())
pipe.add_component("llm", AmazonBedrockChatGenerator(model=""))
pipe.connect("prompt_builder", "llm")- Now you give your prompt and run the pipeline.
variable = "AWS Bedrock"
system_message = ChatMessage.from_system("You are an assistant giving out valuable information to language learners.")
messages = [system_message, ChatMessage.from_user("What is {{ variable }}? How is it used?")]
pipe.run(data={"prompt_builder": {"template_variables": {"variable": variable}, "prompt_source": messages}})
This is a basic tutorial-followed pipeline generation of both the generators supported by Haystack. When using Haystack’s generators, it’s important to note that the framework offers a range of options, as detailed in its documentation. These generators include HuggingFace, Llama, Sagemaker and more.
More tutorials include will walk us through Haystack’s features and functionalities making it easy for us to understand and implement them Lets look at one of these in detail here.
Serializing LLM pipelines
We create a simple pipeline that expects a topic from the user, and generates a summary about the topic with google/flan-t5-large. We are using a local model that we’re getting from Hugging Face. We’re using a relatively small, open-source LLM.
from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators import HuggingFaceLocalGenerator
template = """
Please create a summary about the following topic:
{{ topic }}
"""
builder = PromptBuilder(template=template)
llm = HuggingFaceLocalGenerator(
model="google/flan-t5-large", task="text2text-generation", generation_kwargs={"max_new_tokens": 150}
)
pipeline = Pipeline()
pipeline.add_component(name="builder", instance=builder)
pipeline.add_component(name="llm", instance=llm)
pipeline.connect("builder", "llm")
topic = "Climate change"
result = pipeline.run(data={"builder": {"topic": topic}})
print(result["llm"]["replies"][0])
Haystack supports YAML. Use dumps() to convert the pipeline to YAML.
yaml_pipeline = pipeline.dumps()We get a pipeline YAML that looks like the following. Then we modify the promptbuilder’s template to translate provided sentence to French, and deserialize the pipeline by calling loads()
yaml_pipeline = """
components:
builder:
init_parameters:
template: "\nPlease translate the following to French: \n{{ sentence }}\n"
type: haystack.components.builders.prompt_builder.PromptBuilder
llm:
init_parameters:
generation_kwargs:
max_new_tokens: 150
huggingface_pipeline_kwargs:
device: cpu
model: google/flan-t5-large
task: text2text-generation
token: null
stop_words: null
type: haystack.components.generators.hugging_face_local.HuggingFaceLocalGenerator
connections:
- receiver: llm.prompt
sender: builder.prompt
max_loops_allowed: 100
metadata: {} """
from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators import HuggingFaceLocalGenerator
new_pipeline = Pipeline.loads(yaml_pipeline)
new_pipeline.run(data={"builder": {"sentence": "I love capybaras"}})
We tried a few more like Filtering Documents with Metadata and Preprocessing Different File Types. You can check out more here.
Observed Limitation
Haystack is considered straightforward for its excellent documentation, known for its clarity and ease of comprehension. However, the majority of tutorials and documentation materials regarding generators predominantly revolve around the usage of the OpenAIGenerator component. Therefore, for users seeking consistency with these resources, acquiring an OpenAIGenerator key is advisable for implementing similar use cases. The reason being, while Haystack excels in documentation quality, transitioning to alternative generators for specific use cases covered in provided tutorials may present challenges. The documentation coverage for such transitions might not be as extensive, posing a limitation for users relying solely on Haystack’s guidance. In essence, for users opting for a lighter use of Haystack, adhering to the OpenAIGenerator is recommended to leverage its seamless integration and comprehensive documentation support.
Comparison
LangChain, a general purpose open-source framework, offers value-added services on top of its core to large enterprises, with a rich set of out-of-the-box integration tools including AWS Lambda, APIFY, Huggingface, and YouTube. However, its architecture tends towards complexity, abstracting almost every concept into classes, potentially appealing to users favoring highly object-oriented code bases. The framework revolves around Chains, with Agents facilitating conversation routing and a robust toolkit encompassing loaders, transformers, embedding models, and retrievers. Offering versatility, LangChain provides various options for conversation handling and flexible Output parsers supporting various structures. LangChain’s learning curve is steeper, making it particularly suitable for comprehensive enterprise chat applications.
Haystack, also open-source, is backed by significant funding and supports parent company deepset’s other products like deepset Cloud. It is often opted for lighter tasks or rapid prototyping endeavours. While it offers fewer out-of-the-box integration tools compared to LangChain, Haystack allows users flexibility to craft custom tools like converters, classifiers, retrievers and pipelines like RAG and QA. Renowned for its user-friendly approach, Haystack emphasizes simplicity in understanding and utilizing its components out of the box. Its design focuses on intuitive usage, with integration capabilities extending to REDIS for memory storage across conversations. While its toolset may have slightly fewer features compared to LangChain, Haystack delivers straightforward solutions with a pragmatic approach to NLP tasks.
Conclusion
Haystack is lauded for its simplicity, often favoured for simpler tasks or swift prototypes. With superior documentation quality compared to LangChain, Haystack shines in building large-scale search systems, question-answering engines, summarization, and conversational AI interfaces. Notably, a Retrieval-Augmented Generation (RAG) evaluation revealed Haystack’s superior performance and ease of use, further bolstering its reputation. While Haystack may lack the depth of features found in LangChain, its straightforward nature and proficiency in key NLP tasks make it a reliable choice for many developers. Ultimately, the decision between LangChain and Haystack hinges on specific needs and preferences, balancing factors such as complexity, documentation quality, and integration capabilities.
About the Author:
Anamika Jha is Junior Data Scientist at the Version 1 AI Labs.
