Build LLamaIndex Agents with MCP Connector

Learn how to deploy LLamaIndex Agent workflows with MCP Tools

Setting up MCP server

LLamaIndex MCP Connector

Install MCP python SDK https://github.com/modelcontextprotocol/python-sdk

uv add “mcp[cli]”

Then we can start by writing out simple MCP server in Python. Note that you can use whatever language you prefer for this.

#server.py
from mcp.server.fastmcp import FastMCP


import argparse

mcp = FastMCP("OnBase", port=3000)


@mcp.tool()
def get_document_info(id: int):
    """Gets document information from the client given a document ID.

    Args:
        id: The ID of the document to retrieve.

    Returns:
        dict: Document information retrieved from the client.

    Raises:
        ClientError: If there is an error retrieving the document from the client.
    """
    return {
        "doc_id": id,
        "doc_name": "Document Name",
        "doc_type": "Document Type",
        "doc_size": 123456,
        "doc_date": "2023-10-01",
        "doc_author": "Author Name",
        "doc_description": "Document description goes here.",
        "doc_tags": ["tag1", "tag2", "tag3"],
    }


if __name__ == "__main__":
    # Start the server
    print("🚀Starting server... ")

    # Debug Mode
    #  uv run mcp dev server.py

    # Production Mode
    # uv run server.py --server_type=sse

    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--server_type", type=str, default="sse", choices=["sse", "stdio"]
    )
    print("Server type: ", parser.parse_args().server_type)
    print("Launching on Port: ", 3000)
    print('Check "http://localhost:3000/sse" for the server status')

    args = parser.parse_args()
    mcp.run(args.server_type)

To run the server in debug mode and run MCP Inspector run

uv run mcp dev server.py

You will see a panel like this

MCP Inspector Panel

Now to actually connect this with LlamaIndex let’s deploy it with sse

uv run server.py — server_type=sse

Once you do this you can see see on your http://localhost:3000/sse some sse messages starting to pop up.

Now we are ready to connect it with LLamaIndex.

You can run the MCP client as such

from llama_index.tools.mcp import BasicMCPClient, McpToolSpec

mcp_client = BasicMCPClient("http://127.0.0.1:3000/sse")
mcp_tool = McpToolSpec(client=mcp_client) # you can also pass list of allowed tools

Now e can use that toolspec to get all tools available

tools = await mcp_tool.to_tool_list_async()
for tool in tools:
    print(tool.metadata.name, tool.metadata.description)

Now let’s put it all together and make a quick agent

from llama_index.llms.openai import OpenAI
import dotenv

dotenv.load_dotenv()

# Load LLM
llm = OpenAI(model="gpt-4o")

from llama_index.tools.mcp import McpToolSpec
from llama_index.core.agent.workflow import FunctionAgent, ToolCallResult, ToolCall
from llama_index.core.workflow import Context

SYSTEM_PROMPT = """\
You are an AI assistant for Tool Calling.

Before you help a user, you need to work with tools to interact with Our Database
"""


async def get_agent(tools: McpToolSpec):
    tools = await tools.to_tool_list_async()
    agent = FunctionAgent(
        name="Agent",
        description="An agent that can work with Our Database software.",
        tools=tools,
        llm=llm,
        system_prompt=SYSTEM_PROMPT,
    )
    return agent


async def handle_user_message(
    message_content: str,
    agent: FunctionAgent,
    agent_context: Context,
    verbose: bool = False,
):
    handler = agent.run(message_content, ctx=agent_context)
    async for event in handler.stream_events():
        if verbose and type(event) == ToolCall:
            print(f"Calling tool {event.tool_name} with kwargs {event.tool_kwargs}")
        elif verbose and type(event) == ToolCallResult:
            print(f"Tool {event.tool_name} returned {event.tool_output}")

    response = await handler
    return str(response)


from llama_index.tools.mcp import BasicMCPClient, McpToolSpec


mcp_client = BasicMCPClient("http://127.0.0.1:3000/sse")
mcp_tool = McpToolSpec(client=mcp_client)

# get the agent
agent = await get_agent(mcp_tool)

# create the agent context
agent_context = Context(agent)

Now all that’s left to do is running the Agent

# Run the agent!
while True:
    user_input = input("Enter your message: ")
    if user_input == "exit":
        break
    print("User: ", user_input)
    response = await handle_user_message(user_input, agent, agent_context, verbose=True)
    print("Agent: ", response)

Resources:

LLamaIndex actually introduced a Client for MCP recently. You can check it here: https://docs.llamaindex.ai/en/stable/api_reference/tools/mcp/

That was based on this implementation: https://psiace.me/posts/integrate-mcp-tools-into-llamaindex/

You can find more docs here: https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/tools/llama-index-tools-mcp/examples/mcp.ipynb