How to build an LLM-powered ChatBot with Streamlit

A step-by-step guide using the unofficial HuggingChat API (no APIs required)

Introduction

The remarkable progress made in natural language processing and large language models (LLMs) over the past few months has led to the widespread adoption of generative AI and chatbots. Chatbots that are powered by large language models, like ChatGPT, have become particularly popular.

These chatbots are capable of understanding and mimicking human language, making them invaluable tools for not only businesses but individuals who are also reaping the benefits of these tools as they may help with brainstorming ideas, boost productivity as well as serve as personal or coding assistants.

One of the hurdles of LLM models is that they may quickly consume token credits and for heavily used applications, the costs may add up. In this article, we will build an AI-powered chatbot using an open-source LLM model (OpenAssistant/oasst-sft-6-llama-30b-xor) from the unofficial HuggingChat API known as HugChat. As it is open source, it does not require an API key and thus saves API costs.

In this post, you’ll learn how to:

• Set up the app on the Streamlit Community Cloud
• Build the chatbot

🤗 Want to jump right in? Here's the HugChat app and the GitHub repo.

What the HugChat app can do

Before we proceed with the tutorial, let’s quickly grasp the app’s functionality. Head over to the app and get familiar with its layout — (1) the sidebar provides app info, and (2) the main panel displays conversational messages:

Diagram created by Author.

Interact with it by (1) entering your prompt into the text input box and (2) reading the human/bot messages.

Set up and deploy the app

Clone the app-starter-kit repo to use as the template for creating the chatbot app. Then click on "Use this template":

Give the repo a name (such as mychatbot). Next, click “Create repository from the template.” A copy of the repo will be placed in your account:

Next, follow this blog post to get the newly cloned repo deployed on the Streamlit Community Cloud. When done, you should be able to see the deployed app:

Edit the requirements.txt file by adding the following prerequisite Python libraries:

streamlit
hugchat
streamlit-chat
streamlit-extras

This will spin up a server with these prerequisites pre-installed.

Let’s take a look at the contents of streamlit_app.py:

import streamlit as st

st.title('🎈 App Name')

st.write('Hello world!')

In subsequent sections, you will modify the contents of this file with code snippets about the chatbot.

Finally, before proceeding with app building, let’s take a look at how the user will interact with it:

• Front-end: The user submits an input prompt (by providing a string of text to the text box via st.text_input()), and the app generates a response.
• Back-end: Input prompt is sent to hugchat (the unofficial port to the HuggingChat API) via streamlit-chat for generating a response.
• Front-end: Generated responses are displayed in the app via’s message() command.

Build the chatbot

Fire up the streamlit_app.py file and replace the original content with code snippets mentioned below.

1. Required libraries

Import prerequisite Python libraries:

import streamlit as st
from streamlit_chat import message
from streamlit_extras.colored_header import colored_header
from streamlit_extras.add_vertical_space import add_vertical_space
from hugchat import hugchat

2. Page config

Name the app using the page_title input argument in the st.set_page_config method (it'll be used as the app title and as the title in the preview when sharing on social media):

st.set_page_config(page_title="HugChat - An LLM-powered Streamlit app")

3. Sidebar

Create a sidebar with some information about your chatbot:

with st.sidebar:
    st.title('🤗💬 HugChat App')
    st.markdown('''
    ## About
    This app is an LLM-powered chatbot built using:
    - [Streamlit](<https://streamlit.io/>)
    - [HugChat](<https://github.com/Soulter/hugging-chat-api>)
    - [OpenAssistant/oasst-sft-6-llama-30b-xor](<https://huggingface.co/OpenAssistant/oasst-sft-6-llama-30b-xor>) LLM model
    
    💡 Note: No API key required!
    ''')
    add_vertical_space(5)
    st.write('Made with ❤️ by [Data Professor](<https://youtube.com/dataprofessor>)')

Use the with statement to confine the constituent contents to the sidebar. They include:

• The app title is specified via st.title()
• A short description of the app via st.markdown()
• Vertical space added via add_vertical_space() method from streamlit-extras
• A short credit message via st.write()

4. Session state

Initialize the chatbot by giving it a starter message at the first app run:

if 'generated' not in st.session_state:
    st.session_state['generated'] = ["I'm HugChat, How may I help you?"]

if 'past' not in st.session_state:
    st.session_state['past'] = ['Hi!']

Here, past denotes the human user's input and generated indicates the bot's response.

5. App layout

Give the app a general layout. The main panel will display the chat query and responses:

input_container = st.container()
colored_header(label='', description='', color_name='blue-30')
response_container = st.container()

Use st.container() as a placeholder where the input_container and response_container variables correspond to the human user and chatbot, respectively.

6. Human user input

Create the get_text() custom function that will take prompts provided by the human user as input using st.text_input(). This custom function displays a text box in the input_container:

# User input
## Function for taking user provided prompt as input
def get_text():
    input_text = st.text_input("You: ", "", key="input")
    return input_text

## Applying the user input box
with input_container:
    user_input = get_text()

7. Bot response output

Create the generate_response(prompt) custom function for taking in the user's input prompt as an argument to generate an AI response using the HuggingChat API via the hugchat.ChatBot() method (this LLM model can be swapped with any other one):

# Response output
## Function for taking user prompt as input followed by producing AI generated responses
def generate_response(prompt):
    chatbot = hugchat.ChatBot()
    response = chatbot.chat(prompt)
    return response

Populate the response_container with the AI-generated response with the two underlying if statements:

1. If the user has entered their input query, the if user_input statement will become True and the underlying statements will run.
2. The user-provided prompt (user_input) will serve as an input argument to generate_response() to make the AI-generated response.
3. Subsequently, the generated output will be assigned to the response variable.
4. Both values for user_input and response will be saved to the session state via the append() method.
5. When there are bot-generated messages, the if st.session_state['generated'] statement returns True and the underlying statements will run.
6. A for loop iterates through the list of generated messages in st.session_state['generated']
7. The human (st.session_state['past']) and the bot (st.session_state['generated']) messages are displayed via the message() command from the streamlit-chat component:

## Conditional display of AI generated responses as a function of user provided prompts
with response_container:
    if user_input:
        response = generate_response(user_input)
        st.session_state.past.append(user_input)
        st.session_state.generated.append(response)
        
    if st.session_state['generated']:
        for i in range(len(st.session_state['generated'])):
            message(st.session_state['past'][i], is_user=True, key=str(i) + '_user')
            message(st.session_state['generated'][i], key=str(i))

8. Full code

Putting all of this together, we get the following full code of the app:

import streamlit as st
from streamlit_chat import message
from streamlit_extras.colored_header import colored_header
from streamlit_extras.add_vertical_space import add_vertical_space
from hugchat import hugchat

st.set_page_config(page_title="HugChat - An LLM-powered Streamlit app")

# Sidebar contents
with st.sidebar:
    st.title('🤗💬 HugChat App')
    st.markdown('''
    ## About
    This app is an LLM-powered chatbot built using:
    - [Streamlit](https://streamlit.io/)
    - [HugChat](https://github.com/Soulter/hugging-chat-api)
    - [OpenAssistant/oasst-sft-6-llama-30b-xor](https://huggingface.co/OpenAssistant/oasst-sft-6-llama-30b-xor) LLM model
    
    💡 Note: No API key required!
    ''')
    add_vertical_space(5)
    st.write('Made with ❤️ by [Data Professor](https://youtube.com/dataprofessor)')

# Generate empty lists for generated and past.
## generated stores AI generated responses
if 'generated' not in st.session_state:
    st.session_state['generated'] = ["I'm HugChat, How may I help you?"]
## past stores User's questions
if 'past' not in st.session_state:
    st.session_state['past'] = ['Hi!']

# Layout of input/response containers
input_container = st.container()
colored_header(label='', description='', color_name='blue-30')
response_container = st.container()

# User input
## Function for taking user provided prompt as input
def get_text():
    input_text = st.text_input("You: ", "", key="input")
    return input_text
## Applying the user input box
with input_container:
    user_input = get_text()

# Response output
## Function for taking user prompt as input followed by producing AI generated responses
def generate_response(prompt):
    chatbot = hugchat.ChatBot()
    response = chatbot.chat(prompt)
    return response

## Conditional display of AI generated responses as a function of user provided prompts
with response_container:
    if user_input:
        response = generate_response(user_input)
        st.session_state.past.append(user_input)
        st.session_state.generated.append(response)
        
    if st.session_state['generated']:
        for i in range(len(st.session_state['generated'])):
            message(st.session_state['past'][i], is_user=True, key=str(i) + '_user')
            message(st.session_state["generated"][i], key=str(i))

Wrapping up

In this post, I’ve shown you how to create a chatbot app using an open-source LLM from the unofficial HuggingChat API and Streamlit. You can create your own AI-powered chatbot in only a few lines of code without needing API keys.

I hope this tutorial encourages you to explore the endless possibilities of chatbot development using different models and techniques. The sky is the limit!

If you have any questions, please leave them in the comments below or contact me on Twitter at @thedataprof or on LinkedIn. Share your app creations on social media and tag me, and I’ll be happy to provide feedback or help retweet!

Disclaimer: I work as a Senior Developer Advocate at Snowflake where I create educational content on Streamlit.

Join Medium with my referral link — Chanin Nantasenamat

Read these Next …

Streamlit Quests: Getting Started with Streamlit

How to Master Scikit-learn for Data Science

How to Master Python for Data Science

Watch this Next …

• Streamlit YouTube playlist — A growing collection of 52 Streamlit tutorial videos that I’ve created on my YouTube channel Data Professor.