Building AI Agents with Google ADK, Gemma 3, and MCP Tools

7 min readApr 24, 2025

Introduction

In this article, we’ll explore how to use Google’s Agent Development Kit (ADK) to build agents with local LLMs like Gemma3 or Llama3.2, running via Ollama, and perform real-time search by invoking external tools using Model Context Protocol(MCP).

We will build a YouTube Search Agent using the ADK framework, which will leverage Gemma 3 model for reasoning and response generation, and dynamically call tools via MCP to fetch search results.

What is Gemma 3 Model ?

Gemma 3 , is Google’s latest open-source large language model (LLM), designed to offer high performance, efficiency, and multimodal capabilities.

Gemma 3 is available in multiple sizes (1B, 4B, 12B, and 27B), enabling us to select the ideal model based on our hardware and performance requirements.

Gemma 3 — A collection of lightweight, state-of-the-art open models built from the same research and technology that powers Gemini 2.0 models

Features of Gemma 3 Models

Image and text input: Analyze and interpret visual data with multimodal input support
128K token context: 16x larger input context for analyzing more data and solving more complex problems.
Function calling: Create natural language interfaces to interact with APIs and tools.
Language support: Supports 140+ languages for multilingual AI applications
Model Sizes: Available in 1B, 4B, 12B, and 27B variants to fit our compute and task needs

In the previous article, We explored the ADK framework and MCP Protocol, and demonstrated how to build an agent using ADK as an MCP Client using Gemini-2.5 Pro. Instead of Gemini-2.5 Pro, we will be using the Gemma3 model via Ollama.

Architecture

In this demo, we will build a YouTube Search Agent using the ADK framework, which will leverage Gemma 3 model for reasoning and response generation, and dynamically call tools via MCP to fetch search results.

Core Components :

Google ADK — Provides the agent framework.
Model Context Protocol (MCP) — Standardizes tool communication
Gemma 3 (12B) — Powers the language understanding & generation
Ollama— Hosts the Gemma model locally.
MCP Server — Provides YouTube search capabilities
Python 3.11+ — Base runtime environment

Workflow :

User submits a query through the interface.
ADK Agent Framework processes the query and determines intent
If a search is needed:

Request is routed to the MCP Tool Registry.
MCP Search tool receives the query and fetches results
Results are returned through the MCP standardized format.

4. Gemma 3 model (via Ollama and LiteLlm):

Receives the search results
Generates a natural language response

5. The formatted response is returned to the user interface

Implementation Guide :

Let’s go through a step-by-step breakdown of building local AI assistant using the ADK framework with Gemma 3 model

Pre-Requisites

Python 3.11+ installed
Ollama installed
Download gemma3:12b model.
A valid SerpAPI key (For search)
Install or use any MCP Server local or remote ( mcp-youtube-search )

Project Structure


adk_mcp_gemma3/
├── search/
│   ├── __init__.py              # Package initialization
│   └── agent.py                 # Agent implementation with ADK
├── README.md                    # Project documentation
└── requirements.txt             # Dependencies

Step 1: Setup a virtual environment

Install the dependencies

# Setup virtual environment (Mac or Unix )
python -m venv venv && source venv/bin/active 

# Install agent development kit
pip install google-adk

# Install MCP server 
pip install mcp-youtube-search

# Install litellm
pip install litellm

google-adk: Installs the Agent Development Kit from Google
mcp-youtube-search : Specialized MCP server that handles YouTube searches and fetches results.
litellm: LiteLLM library provides a unified interface to various LLM providers, and in our case, its Ollama

Set Environment variables :

export SERP_API_KEY="your-serpapi-key"

Step 2: Install the MCP Server

For this article, we will use mcp-youtube-search and MCP server, which is built using adk.

# Install from PyPI ( Already Installed from Step 1)
pip install mcp-youtube-search

Step 3: Implement Helper function (agent.py)

# Step 1: Import required modules
from google.adk.agents import LlmAgent
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset, StdioServerParameters
from google.adk.models.lite_llm import LiteLlm
import os
import dotenv

# Step 2: Load environment variables
dotenv.load_dotenv()

# Step 3: Create tool execution helper functions
async def execute_tool(tool, args):
    """Execute a single tool and handle cleanup."""
    try:
        result = await tool.run_async(args=args, tool_context=None)
        return (True, result, None)  # Success, result, no error
    except Exception as e:
        return (False, None, str(e))  # Failed, no result, error message


async def try_tools_sequentially(tools, args, exit_stack):
    """Try each tool in sequence until one succeeds."""
    errors = []
    
    for tool in tools:
        success, result, error = await execute_tool(tool, args)
        if success:
            return result
        errors.append(f"Tool '{tool.name}' failed: {error}")
    
    if errors:
        return f"All tools failed: {'; '.join(errors)}"
    return "No tools available"

LlmAgent — Provides the central agent class that orchestrates the entire system to manage conversation state and history, manage tools, and instruction from the prompt
MCPToolset — Enables connection to and interaction with MCP-compatible tools, acts as an interface for discovering and accessing tools via MCP
StdioServerParameters — Configuration for connecting to an MCP server over standard I/O
LiteLlm — Provides a compatibility layer between ADK and various LLM providers

LlmAgent acts as Orchestrator , LiteLlm as model interface, and MCPToolset as tool registry

Step 4 : Let us create MCP Tool Executor

Below function creates an async wrapper that connects to an MCP tool server and runs available tools — MCPToolset.from_server() to connect to the tool , tries all tools returned by the MCP server sequentially until one succeeds as defined in Step3 and ensures proper resource cleanup with exit_stack.aclose().

# Step 4: Create MCP tool executor
def create_mcp_tool_executor(command, args=None, env=None):
    """Create a function that connects to an MCP server and executes tools."""
    async def mcp_tool_executor(**kwargs):
        # Connect to MCP server
        tools, exit_stack = await MCPToolset.from_server(
            connection_params=StdioServerParameters(
                command=command,
                args=args or [],
                env=env or {},
            )
        )
        
        try:
            # Try all tools until one succeeds
            return await try_tools_sequentially(tools, kwargs, exit_stack)
        finally:
            # cleanup
            await exit_stack.aclose()
    
    return mcp_tool_executor

Step 5 : Create Search Function using above function

We need to ensure proper instruction is provided to search function for LLM to translate natural language query and invoke search function if required.Using mcp-youtube-search MCP Server , tool executor is defined

# Step 5: Create YouTube search function
search_youtube = create_mcp_tool_executor(
    command="mcp-youtube-search",
    args=[],
    env={"SERP_API_KEY": os.getenv("SERP_API_KEY")}
)

# Step 6: Add documentation for the LLM
search_youtube.__name__ = "search_youtube"
search_youtube.__doc__ = """
Search for YouTube videos based on a search query.
    
Args:
    search_query: The search terms to look for on YouTube (e.g., 'Google Cloud Next 25')
    max_results: Optional. Maximum number of results to return (default: 10)

Returns:
    List of YouTube videos with details including title, channel, link, published date, 
    duration, views, thumbnail URL, and description.
"""

Step 6 : Create the LlmAgent

Let us create agent using LlmAgent and Ollama/gemma3:12b

LiteLLM/Ollama KeyError Bug

There’s a known issue with Ollama’s JSON format responses and LiteLLM’s parsing that can cause a KeyError: ‘name’ error.

venv/lib/python3.12/site-packages/litellm/llms/ollama/completion/transformation.py", line 266, in transform_response
    "name": function_call["name"]
KeyError: 'name'

This happens when Ollama returns JSON format that isn’t specifically a tool call format.
Error looks like: KeyError: 'name' in litellm/llms/ollama/completion/transformation.py
I have submitted a bug fix in PR #9966 for the LiteLLM package but is still pending approval and merging.
This bug might cause application to crash or enter an infinite loop when using function calling with Ollama models.

Temporary workarounds:

Manually patch local LiteLLM installation with the changes from PR #9966 and overwrite .venv/lib/python3.11/site-package/litellm/llms/ollama/completion/transformation.py

Above temporary workaround is not required if above bug is merged into litellm

# Step 7: Create the agent with instructions for formatting results
agent = LlmAgent(
    name="youtube_assistant",
    model=LiteLlm(model="ollama/gemma3:12b"),
    instruction="""You are a helpful YouTube video search assistant.
Your goal is to use the search_youtube tool and present the results clearly.

1.  When asked to find videos, call the search_youtube tool.
2.  The tool will return a JSON object. Find the list of videos in the 'results' field of this JSON.
3.  For each video in the list, create a bullet point (*).
4.  Format each bullet point like this: **Title** (Link) by Channel: Description. (Published Date, Views, Duration)
    - Use the 'title', 'link', 'channel', 'description', 'published_date', 'views', and 'duration' fields from the JSON for each video.
    - Make the title bold.
    - Put the link in parentheses right after the title.
5.  Your final response should ONLY be the formatted bullet list of videos. Do not include the raw JSON.
6.  If the 'results' list in the JSON is empty, simply respond: "I couldn't find any videos for that search."
""",
    tools=[search_youtube],
)

Step 7 : Final Demo

Option 1 : ADK Web using Gemma3

adk web

Option 2 : ADK in programmatic Debug mode using Gemma3

Option 3 : ADK CLI using Gemma3

adk run search

Under the hood , Gemma3 uses function calling capability to decides when a tool is needed , ADK provides the framework connecting the model to tools and MCP standardizes how the tools are defined and communicated with MCP Server.

GitHub Repository:

You can access all the code used in this tutorial in my GitHub:

GitHub - arjunprabhulal/adk-mcp-gemma3

Contribute to arjunprabhulal/adk-mcp-gemma3 development by creating an account on GitHub.

github.com

Conclusion

In this article, we demonstrated how to build a local AI agent using Google’s Agent Development Kit (ADK), the Gemma 3 model via Ollama, and external tools connected through the Model Context Protocol (MCP).

Google Cloud - Community