Stream OpenAI respond through FastAPI to Next.js

In ChatGPT have you seen how seen how messages are steam through as they are generated. So you don’t have to wait until the LLM to generate whole respond before you receive it.

This is can be easily done by the API provided with OpenAI. But what if we need to use a backend and stream that respond through it.

For this we use below technologies,

OpenAI FastAPI Next.js

New Code Version available at

Stream OpenAI respond through FastAPI to Next.js 14— Better Version

Setting OpenAI with Python

To achieve the streaming through OpenAI API, we need to enable stream=True in chat complexation API.

Note that in here i have given question as input for the function.

You can check this function by passing text and in your IDE it should stream the result.

def get_openai_generator(question: str):
    openai_stream = openai.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {
                "role": "system",
                "content": f"Question: {question}…