Build a Chatbot for any Database in Minutes with Google GenAI Toolbox & LlamaIndex & Gemini

Introduction

What if we could manage our entire database just by asking questions in plain English?

In this article, I’ll walk you through how I used Google GenAI Toolbox, LlamaIndex AgentWorkflow, and Google Gemini 1.5 Pro to build a conversational assistant that talks to a real PostgreSQL database (Pagila schema) — and runs movie rentals, searches, and customer lookups using natural language.

Google recently announced public beta launch of Gen AI Toolbox for Databases in partnership with LangChain (orchestration framework for building LLMs)

Image by Author : ChatBot Using GenAI ToolBox + LlamaIndex Agent + Gemini Pro LLM

What is Gen AI Toolbox for Databases ?

• Open-source tool to connect GenAI models with databases like PostgreSQL and MySQL
• Enables natural language queries over databases — no SQL needed
• Uses YAML files to define tools like “search films” or “get customer rentals” (as shown in this demo)
• Works with LLMs like Gemini to pick and run tools based on user queries
• Includes observability features via OpenTelemetry for monitoring and debugging
• Supports many databases: PostgreSQL, MySQL, Cloud SQL, Spanner, AlloyDB, and more
• Integrates seamlessly with LLM frameworks like LlamaIndex and LangChain

Image from https://googleapis.github.io/genai-toolbox/getting-started/introduction/

GenAI Toolbox acts as a bridge between our agent orchestration framework (like LlamaIndex) and our database (like PostgreSQL). It provides a centralized control plane where we can define, update, and manage tools — without needing to redeploy our entire application.

What is LlamaIndex AgentWorkflow?

The AgentWorkflow is an orchestrator for running a system of one or more agents.

• Simplifies orchestration of AI agents
• Maintains context and state across steps
• Supports single and multi-agent workflows
• Built on top of LlamaIndex Workflows
• Enables real-time monitoring and streaming
• Reduces boilerplate and accelerates dev time
• Easy to integrate into existing apps

Architecture Overview

DVD Rental Assistant ChatBot is built using a 3-tier architecture( Backend /Frontend/Database along with AI layer . Each component has a distinct role, enabling natural language interactions with a PostgreSQL database through secure, intelligent orchestration using LLM framework

1. Frontend: Streamlit
2. Backend: FastAPI
3. LlamaIndex AgentWorkflow
4. GenAI Toolbox
5. Pydantic
6. Gemini Pro

End to End Workflow:

1. User enters a query in Streamlit.

2. Streamlit sends it to the FastAPI backend.

3. FastAPI invokes the LlamaIndex AgentWorkflow.

4. Agent decides: use Gemini directly or call a YAML-defined tool.

5. If tool is needed, it’s executed via GenAI Toolbox.

6. Results are interpreted and sent back through the stack to the user.

Implementation Guide

Let’s go through a step-by-step breakdown of building this DVD Rental Assistant Conversation ChatBot:

Pre-Requisites

• Python 3.9+
• PostgreSQL with Pagila database
• Google API key for Gemini
• GenAI Toolbox access

Step 1 : Setting up Environment

Install the Python 3.9+ and Setup the virtual env.

python -m venv venv
source venv/bin/activate

Step 2 : Clone github repository

git clone https://github.com/arjunprabhulal/gemini-toolbox-dvd-rental-assistant
cd gemini-toolbox-dvd-rental-assistant
pip install -r requirements.txt

import asyncio, os, logging
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from llama_index.core.agent.workflow import AgentWorkflow
from llama_index.llms.google_genai import GoogleGenAI
from toolbox_llamaindex import ToolboxClient

• Loads all required modules:
• FastAPI for backend API
• Pydantic for data validation
• ToolboxClient to connect to GenAI Toolbox
• GoogleGenAI to load Gemini 1.5 Pro model
• AgentWorkflow from LlamaIndex to handle the orchestration

Step 3: Setting up Google Project and API Key from Gemini

Create Google Cloud Project and enable VertexAI library and initialize the agent

vertexai_config: Contains:

• “project”: Your Google Cloud project ID where Vertex AI is enabled [ vertex-ai-experminent is my personnel GCP project ]
• “location”: The GCP region where the model is deployed (e.g., us-central1)

llm = GoogleGenAI(
    model="gemini-1.5-pro",
    vertexai_config={"project": "vertex-ai-experminent", "location": "us-central1"},
)

cp .env.example .env

cat .env
# Gemini API Configuration
#GOOGLE_API_KEY=your_gemini_api_key_here

# PostGreSQL Database Configuration
DB_HOST=localhost
DB_PORT=5432
DB_NAME=toolbox_db
DB_USER=toolbox_dbuser
DB_PASSWORD=<your-db-password.

Step 4 : Installing Gen AI ToolBox Setup

Download 0.2.1 version toolbox for this demo

export VERSION=0.2.1
curl -O https://storage.googleapis.com/genai-toolbox/v$VERSION/darwin/arm64/toolbox
chmod +x toolbox
./toolbox --version

Step 5 :Database Setup

We are using the Pagila database — an example schema designed for PostgreSQL, inspired by the Sakila schema for MySQL.

Download PostGresSQL from PostgreSQL 16+ and the psql client

# Step 1: Create and load database
psql -U postgres
CREATE DATABASE toolbox_db;
\c toolbox_db
\q

# Step 2: Download Pagila schema and data
mkdir -p database/pagila
cd database/pagila
curl -O https://raw.githubusercontent.com/devrimgunduz/pagila/master/pagila-schema.sql
curl -O https://raw.githubusercontent.com/devrimgunduz/pagila/master/pagila-data.sql

# Step 3: Load Pagila into the toolbox_db
psql -U postgres -d toolbox_db -f pagila-schema.sql
psql -U postgres -d toolbox_db -f pagila-data.sql

Once database and schema is initialized and loaded, we can verify schema

Step 6 : Define Datamodel for Pydantic

FastAPI uses for data validation and documentation

ChatRequest — Incoming User Message and represents the structure of data sent from the frontend to the backend.

class ChatRequest(BaseModel):
    message: str
    user_id: str = "user"

ChatResponse — AI reply to the User and defines the format of the response returned by the /chat endpoint.

class ChatResponse(BaseModel):
    response: str

Step 7: Initialize FastAPI

app = FastAPI(title="DVD Rental Assistant API", description="API for DVD rental operations powered by Google Gemini")

Step 8 : LlamaIndex Agent Setup and ToolBoxClient setup

Initialize Gemini 1.5 Pro using GoogleGenAI

Connects to GenAI ToolBox Server (localhost:5000)

Loads YAML defined tool from `dvdrental_tools.yaml` file

• Creates an agent using AgentWorkflow.from_tools_or_functions() to:
• Route queries
• Choose appropriate tools
• Return Gemini-powered responses

Injects a custom system prompt to shape the agent’s behavior

def get_agent():
    """Creates and configures the LlamaIndex agent with Google Gemini"""
    llm = GoogleGenAI(
        model="gemini-1.5-pro",
        vertexai_config={
            "project": "vertex-ai-experminent",
            "location": "us-central1"
        }
    )

    client = ToolboxClient("http://127.0.0.1:5000")
    tools = client.load_toolset()

    return AgentWorkflow.from_tools_or_functions(
        tools,
        llm=llm,
        system_prompt=prompt
    )

`dvdrental_tools.yaml — `Define all our tool based on our Prompt definition request and response . Example of this demo looks like below

sources:
  my-pg-source:
    kind: postgres
    host: 127.0.0.1
    port: 5432
    database: toolbox_db
    user: toolbox_user
    password: my-password
tools:
  list-all-films:
    kind: postgres-sql
    source: my-pg-source
    description: List all films in the inventory.
    statement: SELECT film_id, title, release_year, length, rating FROM film ORDER BY title LIMIT 50;
..
..
...
....

Step 9 : Start all Services

Start Backend , Frontend and GenAI ToolBox and ensure Psql is up and running

# Start GenAI Toolbox
./toolbox --tools_file "dvdrental_tools.yaml"

# Start Backend
uvicorn backend:app --reload

# Start Frontend
streamlit run streamlit_app.py

Verify able to access applications

GenAI Toolbox : http://locahost:5000
Backend: http://localhost:8000
Frontend: http://localhost:8501

Step 10 : Demo

• The database contains 1,000+ films across 16 unique categories

• The user starts by requesting to rent a specific movie

ChatBot Flow:

1. Prompt for customer ID
2. If not found → prompt for customer details and register
3. Check film availability → register rental
4. Update rental history and respond with confirmation

Showing rental history fetches from PostGreSQL

Verify if data is persisted in DB from Chatbot conversation

Common Issue: Rate Limiting

During frequent interactions, Gemini occasionally returns a 429 Too Many Requests error due to API rate limits. To handle this gracefully, I implemented an exponential backoff retry mechanism with a 3-second initial delay, allowing the assistant to recover and respond without crashing.

GitHub Repository

You can find the complete code and demo here: GenAI-Toolbox-dvd-rental-chatbot

Conclusion

GenAI Toolbox for Databases is a powerful framework that bridges the gap between natural language and structured data operations. By combining Google Gemini, LlamaIndex AgentWorkflow, and YAML-defined SQL tools, it enables seamless, conversational database experiences.

While this demo focused on DVD rentals, the same setup can be extended to:

• Healthcare — patient records, appointments
• E-commerce — customer support bots, order tracking
• Finance — transaction summaries, account queries
• Logistics — inventory and order management