How to Create an AI-Powered Website Analysis Chatbot

Overview

Analyzing websites and extracting relevant information is super important in today's digital age. So many of us need help finding what we need among all the web data. That's where SiteSleuth comes in—an “AI-powered Website Analysis Chatbot” that efficiently processes web content and gives you the answers you're looking for. This blog post will teach you how to create a chatbot to analyze websites and share valuable insights. It's perfect for developers familiar with Python and basic AI concepts like RAG (Retrieval Augmented Generation), Gemini API calls, and more. By the end of this post, you'll understand how SiteSleuth, an AI-powered chatbot, is created to analyze website content and respond to user queries. Thus, you will gain a clear understanding of the steps involved and the skills you need to undertake a similar project.

Prerequisites

Before proceeding with the rest of the article, ensure that you have the necessary knowledge and tools:

Knowledge

• Python Programming: Essential for understanding and modifying the project’s source code.
• API Integration: A basic understanding of how to make API calls, specifically using the Gemini API.
• Cloud Computing Basics: Understanding cloud computing concepts will help in deploying and managing the application on Google Cloud.
• Retrieval Augmented Generation (RAG): Familiarity with RAG concepts will help understand the implementation using Langchain.

Tools

• Google Cloud Platform (GCP) Account: Needed to use and access Google Cloud services, such as Cloud Run for deployment, among others. You can refer to this page if you do not already have a billing account.
• Google Gemini API Key: Required to access and utilize the Gemini models for large language models (LLM). To get your own Gemini API key, you can access it here in Google AI Studio.
• A browser, such as Chrome or Firefox.

Design

In this section, we will outline the high-level architecture of the solution, detailing the rationale behind the design and its impact on usability and functionality. Every time we develop our application, we will review our diagram. However, before that, let us first take a high-level look at the concept of RAG.

There are the following key stages within RAG, which will be part of most applications:

Stages within RAG (LlamaIndex docs)

• Loading: This involves obtaining data from various sources, such as text files, PDFs, websites, or databases, and bringing it into the pipeline.
• Indexing: This involves creating a data structure that allows the data to be easily searched and queried. For LLMs (Language Model Models), this usually means creating vector embeddings to enable accurate and contextual retrieval of data.
• Storing: After data is indexed, it is important to store the index and other metadata to avoid the need for re-indexing.
• Querying: With any indexing strategy, LLMs can be used to query the data in various ways.
• Evaluation: Evaluation provides objective measures of how accurate and reliable the responses to queries are.

After understanding the stages of RAG and their respective functions, it’s time to review the application diagram that we will use to develop the application. We will thoroughly examine all the annotated parts of the diagram, as we will be implementing them later. Here’s a breakdown:

Retrieval Augmented Generation (RAG) Application Architecture

1. User Input: A user submits a prompt or query (step 1) via an interface, such as a website.
2. Semantic Search: The system translates the prompt into a format suitable for semantic search (step 2).
3. Retrieval System: This component conducts a search within a document repository (step 3) based on the semantically processed prompt.
4. Relevant Documents: The retrieval system identifies and retrieves relevant documents from the repository (step 4) based on their semantic similarity to the prompt.
5. Contextual Information: The retrieved documents, containing relevant information, are prepared as contextual information (step 5) to be fed to a Large Language Model (LLM).
6. LLM Processing: The prepared prompt, along with the retrieved contextual information, is passed to the LLM. The LLM processes this input and generates a response (step 6).
7. Generated Response: The generated response from the LLM is then presented back to the user (step 7) through the interface.

This is mostly about our application architecture. But what about deployment? Let’s talk about it then.

Deploying and scaling applications with Cloud Run

The image depicts a simple workflow for deploying and scaling an application using containers and a serverless platform like Google Cloud Run. Here’s a breakdown of each step:

1. Write your code: The process starts with writing the RAG application code based on the high-level architecture we discussed earlier.
2. Package it into a container: The code is then packaged into a container, ensuring the application code is encapsulated for portability and dependency management.
3. Deploy on Cloud Run: The containerized application is deployed to Cloud Run, a serverless platform that automatically scales containers based on demand. Cloud Run adjusts the number of running container instances based on traffic, ensuring optimal resource utilization and performance. It also handles traffic routing and distributes incoming requests across the running container instances.

That mostly includes the high-level architecture of the solution. Next, we will build the application for you with step-by-step instructions.

Step-by-step instructions

To develop the application, we will divide the entire process into small, manageable tasks. As each task is completed, we will progress towards building and deploying the application. Here is the order of our tasks:

Task 0. Setup and requirements

Before building the application, make sure you have the following setups and requirements ready:

Create a project

1. In the Google Cloud Console, navigate to the project selector page and choose or create a Google Cloud project.

GCP Project Creation

2. Make sure billing is enabled for your cloud project. Find out how to check if billing is enabled on a project.

3. You will use Cloud Shell, a command-line environment that runs in Google Cloud. Click "Activate Cloud Shell" at the top of the Google Cloud console.

4. Once you are connected to Cloud Shell, you can verify that you are already authenticated and that the project is set to your project ID by using the following command:

gcloud auth list

Output:

Credentialed Accounts

ACTIVE: *
ACCOUNT: <Your email registered to GCP>

To set the active account, run:
    $ gcloud config set account `ACCOUNT`

5. Run the following command in Cloud Shell to ensure that the gcloud command recognizes your project.

gcloud config list project

Output:

[core]
project = <Your GCP project name>

6. If your project is not configured, use the following command to configure it:

gcloud config set project <YOUR_PROJECT_ID>

7. Enable the required APIs. You can use the gcloud command in the Cloud Shell terminal:

gcloud services enable artifactregistry.googleapis.com cloudbuild.googleapis.com run.googleapis.com logging.googleapis.com pubsub.googleapis.com iam.googleapis.com cloudaicompanion.googleapis.com --project PROJECT_ID

8. Make sure to enable the required APIs. You can use the gcloud command or access them through the console by searching for each product.

For information on gcloud commands and usage, please refer to the documentation.

Task 1. Code Understanding (Optional)

Let us take a look at the codebase to get a better understanding of the project structure and each component’s functionality. Here’s an overview of the important directories and files:

Folder Structure & Functionality

Understanding these components will allow you to navigate and modify the code more efficiently. While this task is optional, it is strongly advised for anyone who wishes to extend or customize the functionality of the application.

Task 2. Run the Application Locally

Now, it’s time to run the application. First, check that everything is working as expected, and then move on to the deployment phase. Here is a guide to running the application locally:

Clone the application

1. Open a new Cloud Shell terminal by clicking on the Cloud Shell icon located in the top right corner of the Cloud console.

git clone https://github.com/sitamgithub-MSIT/SiteSleuth.git

2. Run the following commands to clone the repo and navigate to the SiteSleuthdirectory in Cloud Shell using the following commands:

cd SiteSleuth

To run the Streamlit Application, you will need to perform some additional steps.

Run the application

1. Setup the Python virtual environment and install the dependencies:

python3 -m venv sitesleuth-env
source sitesleuth-env/bin/activate
pip install -r requirements.txt

2. Your application requires access to two environment variables:

• GOOGLE_API_KEY: This is the Google Gemini API key.
• USER_AGENT: This is the name of the user agent. E.g. SiteSleuth v1.0.

These variables are necessary for langchainand google-generativeaipackage. The specific code line from the src/config.py the function is shown here: genai.configure(api_key=os.environ.get("GOOGLE_API_KEY))

Click on “Open editor” in the cloud shell and create a .envfile within the SiteSleuthproject directory to store the environment variables.

GOOGLE_API_KEY=YOUR_GEMINI_API_KEY
USER_AGENT=YOUR_USER_AGENT_NAME

Creating an environment file

Otherwise, you can execute the following command in the Cloud Shell:

export GOOGLE_API_KEY='<Your Gemini API key>' # You can get the API key from Google AI Studio.
export USER_AGENT='<Your User Agent Name>'  # Example: SiteSleuth v1.0

3. To run the application locally, execute the following command:

streamlit run app.py \
  --browser.serverAddress=localhost \
  --server.enableCORS=false \
  --server.enableXsrfProtection=false \
  --server.port 8080

Output:

Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.

You can now view your Streamlit app in your browser.

URL: http://localhost:8080

4. The application will start, and you will be provided with a URL to access it. Click the link to open the application in the browser, or use Cloud Shell’s web preview function to launch the preview page.

5. Navigate back to Cloud Shell and authorize the application. Once you have authorized the application, you can navigate back to the application to see the response.

Response Generation (Running Locally)

6. After you have finished testing the application, you can stop the application by entering Ctrl + C in Cloud Shell.

Task 3. Build and Deploy the Application to Cloud Run

As everything is functioning well locally, it's time to deploy our application to Cloud Run. Here are step-by-step instructions on how to do so.

1. For deployment, you will need environment variables. We already have the GOOGLE_API_KEY and USER_AGENT environment variables set in the previous section. If you have not set them, you can set them in a .envfile.

GOOGLE_API_KEY='Your Gemini API key' # You can get the API key from Google AI Studio.
USER_AGENT='Your User Agent Name'  # Example: SiteSleuth v1.0

2. The next and final step is to deploy the service on Cloud Run.

gcloud run deploy

3. After running the command, it will prompt you to enter the source code location. Since we are already in the project directory, no input is necessary.

4. Then, it will ask for the service name. Please provide the service name as "sitesleuth-app”.

Deployment Image-1

5. Then it will inquire about the deployment region. Select the region us-central1from the provided list.

6. To build the Docker image for the application, you need to push it to the Artifact Registry. Since we don’t have one, you must create an Artifact Registry repository. Cloud Shell will prompt you to create an artifact registry, so putY.

7. It will ask you about the unauthenticated invocations(y).

Deployment Image-2

8. After this, you will finally see the process of building using a Dockerfile and deploying the container to the Cloud Run service in the project’s region. In your case, it is us-central1 .

Deployment Image-3

Note: Please be patient, as this process will take some time. The output will be ready shortly.

Output:

Deployment Image-4

Building using Dockerfile and deploying container to Cloud Run service [sitesleuth-app] in project [sitesleuth-test] region [us-central1]
OK Building and deploying new service... Done.                                                                                                                                                   
  OK Creating Container Repository...                                                                                                                                                            
  OK Uploading sources...                                                                                                                                                                        
  OK Building Container... Logs are available at [https://console.cloud.google.com/cloud-build/builds/7a9e1157-7536-498f-a608-46cca63621f5?project=938577189326].                                
  OK Creating Revision...                                                                                                                                                                        
  OK Routing traffic...                                                                                                                                                                          
  OK Setting IAM Policy...                                                                                                                                                                       

You will receive a URL for the Cloud Run service on successful deployment. You can open it in your browser to see the deployed Cloud Run application.

Output:

Deployment Image-5

Done.
Service [sitesleuth-app] revision [sitesleuth-app-00001-xlr] has been deployed and is serving 100 percent of traffic.
Service URL: https://sitesleuth-app-36j5wkpwpa-uc.a.run.app

Troubleshooting

Some common issues that one might face while running such Generative AI projects are:

1. Wrongly set environment variables, especially for the API keys. Make sure to have a valid Gemini API key configured. If you are using the Google Cloud Secret Manager, make sure to have the correct IAM permissions to access the secret.
2. The next one is not an error but a common issue: the model might not generate the expected response. This can be a poor configuration of the model configurations. For example, you can change the max tokens, temperature, etc. to get better results. Or for Gemini, you can switch to a different model, like “gemini-1.5-flash”. Here is the link to the available model variants.
3. While deploying the model to Cloud Run, we are creating an artifact registry. You can verify the image in the Artifact Registry by visiting the link and selecting the project and repository.

Artifact Registry on Google Cloud

4. Similarly, you can verify the Cloud Run service by visiting the link and selecting the project and service.

Cloud Run Service

5. For most of the troubleshooting, you can refer to the logs in the Cloud Run service. We already enabled the Google Cloud logging api in our project. You can view the logs by visiting the link for your project.

Optional (Troubleshooting):

More advanced troubleshooting can be done by actually enabling Error reporting. You can set up error reporting by visiting the link for your project. Although it is out of the scope of this blog. But it can be really helpful in debugging the issues. Specifically, with our custom exception handling, error reporting works like a charm.

Reporting errors

As you can see in the above image, with the error reporting and our custom exception handling, we can see the error message and the stack trace. And even which file and line number the error occurred. This can help debug the issues. That’s all about the troubleshooting.

Clean up

After testing the application and checking your results with the demo section provided later, follow these steps to avoid incurring charges to your Google Cloud account for the resources used in this post:

1. In the Google Cloud console, go to the Manage Resources page.
2. In the project list, select the project you want to delete and then click Delete.
3. To delete the project, type the project ID into the dialog and then click Shut down.

Congratulations! 🎉

Demo

By the end of the above section's success, you will have a fully functional chatbot capable of analyzing website content and providing detailed answers to user queries. Below are images of the final demo.

Demo Image-1

In the Streamlit application, I added the link to the Hugging Face blog about Google’s recent release of the open-source large language model “Gemma 2.” After clicking the submit button, the application processed the web data and displayed “done” on the left-hand sidebar, indicating that it was ready for the user to submit their query and receive answers based on it.

Demo Image-2

Later, I asked some questions from the blog about the advancements of the Gemma-2 model compared to its previous version, Gemma-1. I received a well-organized answer in point format, providing information about all the advancements.

Demo Image-3

You can try the same with any website link you prefer. By analyzing the website content, the chatbot will attempt to provide an answer to your query. However, there may be cases where the chatbot cannot provide an answer based on your given context. For example, if you look at the image above, it answers my questions about the Gemma blog. However, when I ask about Gemma-2-2B, it can't provide an answer. That's because Gemma-2 2B was not released at that time, so it's not within the context. Therefore, the model naturally responds that the information is not in the provided context.

What’s next?

To further enhance your skills and explore the capabilities of SiteSleuth, consider the following:

Future Possibilities

• Explore other vector stores and data loaders for the application. Here are some resources: “LangChain Document Loaders for Web Data” and Vector Stores Collection for LangChain.
• Adding the Cloud Build (CI/CD) solution to the Cloud Run deployment. Here’s an example for reference.
• Add “tracing” to the project to understand the behavior of the LLM application. Explore LangSmith documentation for tracing.

Resources

• Find out more about Generative AI on the Google Cloud Tech YouTube channel.
• Google Cloud Generative AI official repo
• Example Gemini notebooks

Call to action

To learn more about Google Cloud services and to create an impact for the work you do, get around to these steps right away:

• Register for Code Vipassana sessions
• Join the meetup group Datapreneur Social
• Sign up to become a Google Cloud Innovator