LangServe: Deploying Language Models Made Easy

Introduction

Language models have transformed the way we interact with computers. From chatbots to virtual assistants, these models are the brains behind many of our favorite applications. But taking an idea from a prototype to a real-world application can be a daunting task, especially if you’re not a machine learning expert. That’s where LangServe comes in.

What is LangServe?

LangServe is a powerful tool that simplifies the deployment of language models. It’s like turning your language model prototype into a real, working application. Think of it as the bridge that connects your brilliant idea with the people who can benefit from it.

Why LangServe Matters

Imagine you’ve built a chatbot that can help answer questions. You’ve created it in a notebook, tested it, and it works like a charm. But now, you want to share it with the world. That’s where LangServe comes in. It takes your prototype and turns it into a full-fledged application.

Here’s why this is a game-changer:

1. Fast and Easy Deployment

LangServe makes deploying your language model quick and painless. You can go from a simple prototype to a real application that’s ready for users in no time.

2. No Coding Hassles

You don’t need to be a coding guru to use LangServe. It’s designed to be user-friendly and doesn’t require a deep understanding of complex programming.

3. Scaling Made Simple

LangServe ensures your application can handle multiple requests simultaneously. It’s ready for production-level usage without any extra headaches.

4. Intermediate Results Access

Sometimes, you might want to see what’s happening inside your application, even before the final result is ready. LangServe offers a way to check these intermediate steps, making it a great tool for debugging and improving your application.

How Does It Work?

Using LangServe is straightforward. You create your language model and pass it to LangServe. Then, with just a few lines of code, you get a web server that can handle incoming requests.

Examples of LangServe in Action

Let’s say you built a chatbot that can tell you all about bears. With LangServe, you can turn this chatbot into a real website or application that people can access. You can send a question about bears, and the chatbot will provide you with answers in real-time.

#This is the my_package/chain.py file.

# A conversational retrieval chain using LangChain
from langchain.chains import ConversationalRetrievalChain
from langchain.chat_models import ChatOpenAI
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS

# Create a vector store
vectorstore = FAISS.from_texts(["cats like fish", "dogs like sticks"], embedding=OpenAIEmbeddings())
retriever = vectorstore.as_retriever()

# Create a ChatOpenAI model
model = ChatOpenAI()

# Create the ConversationalRetrievalChain from the model and retriever
chain = ConversationalRetrievalChain.from_llm(model, retriever)

#This is the my_package/server.py
# A server for the chain
from fastapi import FastAPI
from langserve import add_routes
from my_package.chain import chain

app = FastAPI(title="Retrieval App")

# Add the LangServe routes to the FastAPI app
add_routes(app, chain)

if __name__ == "__main__":
    import uvicorn

    # Run the FastAPI app
    uvicorn.run(app, host="localhost", port=8000)

# Deploy your LangServe app on GCP Cloud Run
# Replace the URL with your deployment URL
curl https://langserve-launch-example-vz4y4ooboq-uc.a.run.app/stream -X POST -H "Content-Type: application/json" --data '{"input": {"topic": "bears"}}'

These code snippets demonstrate how to create a Conversational Retrieval Chain, set up a FastAPI server, and make API requests to your LangServe application. This is how you turn your language model prototype into a scalable, production-ready application.

What’s Next for LangServe

We’re continually improving LangServe to make it even more user-friendly. In the coming weeks, we plan to add more features, like a playground for experimenting with different prompts and retrievers. We also want to enable you to save multiple configurations for the same chain, making it more versatile and adaptable.

Conclusion

LangServe takes the headache out of deploying your language model applications. You don’t need to be a coding expert to make your brilliant ideas accessible to the world. Whether it’s a chatbot or a virtual assistant, LangServe can help you bring your language model to life.