Multi AI agent systems with Crew AI

In this tutorial, we’ll break down the complex process of creating a multi-agent vacation planner using Crew AI.

We’ll cover the key concepts of agents, tools, tasks, and the crew, explaining how they interact to deliver a cohesive solution. I encourage you to follow along with the example code in the GitHub repository to equip yourself to create your own intuitive Crew AI system.

This guide assumes some familiarity with Python programming and basic concepts of AI and automation.

Introduction to AI Agents

AI agents are autonomous entities designed to perform specific tasks.

In our scenario, multiple agents automate vacation planning, each handling roles like finding accommodation, planning itineraries, and managing the budget.

What is Crew AI?

Crew AI is a framework that simplifies managing multiple AI agents working together on complex tasks.

It leverages AI to distribute responsibilities among specialized agents, each with specific tools and capabilities. Key components include agents, tools, tasks, and the crew itself.

Crew AI’s execution flow

Crew AI breaks down complex problems into manageable tasks, each handled by specialized agents. Here’s a step-by-step overview of how it functions:

1. Set up tools: Agents need predefined tools like web scrapers or data-fetching APIs to achieve their goals. These tools help agents gather information, process data, and interact with external systems.
2. Define agents and roles: Agents are autonomous units that perform specific roles within a larger task using predefined goals and tools. Define these agents by assigning them specific goals and providing the necessary tools to achieve those goals.
3. Create tasks: A task defines what an agent needs to accomplish, including a detailed description, expected output, and any necessary input parameters. Tasks are crucial in guiding agents towards their goals.
4. Assemble the crew: Agents are assembled into a crew, which manages the process flow — typically using a higher level language model (LLM). It ensures that agents collaborate effectively and complete their tasks in a coordinated manner.
5. Execute Tasks: The crew kicks off the process by executing the defined tasks. Each agent works on its assigned task, using the provided tools to gather information, process data, and generate results.
6. Coordinate and integrate: The LLM coordinates the activities of the agents, ensuring that tasks are completed in a logical sequence and that the output of one agent can be used as input for another.
7. Generate final output: Once all tasks are completed, the crew compiles the results into a final output.

Why Use Crew AI?

Crew AI provides significant advantages for automating complex tasks. By distributing responsibilities among multiple agents, it enhances scalability and efficiency compared to a single monolithic system.

Each agent is specialized in a specific role, ensuring more effective and accurate task execution. The framework also facilitates seamless collaboration, integrating agents’ efforts to produce a cohesive final output.

Additionally, Crew AI is highly flexible, making it adaptable to various use cases beyond vacation planning, such as event organization, customer support, and data analysis.

Defining tools for agents

Tools provide the necessary capabilities for agents to gather, process, and interact with data, making it possible for them to achieve their goals.

In our automated vacation planner, tools play a crucial role in enabling agents to search for information, scrape data from websites, and manage the gathered information effectively.

The importance of tools

Tools extend the capabilities of agents by providing specialized functions that are needed to accomplish specific tasks.

Here’s why tools are so important:

1. Specialization: Tools provide specialized functions that cater to specific needs, such as web scraping, data searching, or API interaction.
2. Efficiency: Tools automate repetitive processes and handle complex operations, saving time and reducing the risk of errors.
3. Integration: Tools enable agents to interact with various data sources and systems.

How tools work

Tools work by providing specific functionalities that agents can call upon during task execution.

Agents use these tools by invoking their functions, passing necessary parameters, and handling the returned data.

Examples of tools in our project

For our vacation planner project, we use two built-in tools:

• SerperDevTool: This tool is used for searching the web. It can perform searches based on specified criteria and return relevant results that the agents can use.
• ScrapeWebsiteTool: This tool is used for scraping data from websites. It extracts specific information from web pages, which the agents can then process and utilize in their tasks.

Here’s how we define and initialize these tools to be used later:

from crewai_tools import ScrapeWebsiteTool, SerperDevTool

# Initialize the tools
search_tool = SerperDevTool()
scrape_tool = ScrapeWebsiteTool()

Creating agents

Agents are the core components that carry out tasks in Crew AI. Each agent is designed to perform a specific role within the overall task, utilizing various tools to accomplish its goals.

Defining agents and their roles

In Crew AI, agents are specialized units that focus on particular aspects of a complex task.

For our automated vacation planner, we define three agents: a rental coordinator, an itinerary planner, and a budgeting agent. Each agent has a distinct role, set of goals, and tools it uses to accomplish these goals.

The backstory gives the agent some context about what their role entails, and guidance on how to complete it.

Agent 1: Rental Coordinator

The Rental Coordinator is responsible for identifying suitable accommodation options. This agent searches for hotels, rental homes, or vacation rentals that fit the specified criteria such as location, dates, group size, and budget.

# Define the Venue Coordinator agent
rental_coordinator = Agent(
   role="Hotel/Rental Coordinator",
   goal="Identify an appropriate hotel, rental home, or vacation rental.",
   tools=[search_tool, scrape_tool],
   verbose=True,
   backstory=(
     "With a keen sense of space and understanding of vacation logistics, "
     "you excel at finding and securing the perfect vacation rental that fits "
     "the vacation's location, start and end dates, group size, and budget constraints."
    )
)

Agent 2: Itinerary Planner

The Itinerary Planner focuses on creating a detailed itinerary, including daily excursions and activities. This agent takes into account the traveler’s interests, budget, and logistical needs such as transportation.

# Define the Itinerary Planner agent
itinerary_planner = Agent(
   role="Itinerary Planner",
   goal="Create a proposed itinerary including daily excursions and activities.",
   tools=[search_tool, scrape_tool],
   verbose=True,
   backstory=(
     "With a passion for adventure and local culture, you specialize in planning engaging "
     "and budget-friendly itineraries, taking into account the traveler's interests, budget, "
     "and logistics like transportation needs."
    )
)

Agent 3: Budgeting Agent

The Budgeting Agent is tasked with managing the overall budget for the trip. This agent ensures that all aspects of the vacation, including accommodation and daily activities, stay within the allocated budget.

# Define the Budgeting Agent
budgeting_agent = Agent(
   role="Budgeting Agent",
   goal="Manage the overall budget for the trip, considering the cost of accommodation and daily activities.",
   tools=[search_tool],
   verbose=True,
   backstory=(
     "With a knack for financial planning, you ensure the vacation remains within budget while maximizing value and enjoyment."
   )
)

Understanding the code

Agent Definition:

• Each agent is defined with a role, goal, and a set of tools.
• The verbose parameter is set to True to enable detailed logging, which is useful for understanding the agent’s actions and debugging.
• The backstory parameter provides additional context and motivation for the agent’s behavior, enhancing its decision-making process.

Specific Roles:

• The Rental Coordinator focuses on finding accommodation that meets specific criteria.
• The Itinerary Planner creates a daily itinerary, considering various factors such as traveler interests and budget.
• The Budgeting Agent manages the financial aspects of the vacation, ensuring all plans stay within budget.

By defining these agents with specific roles and tools, we create a system where each component works autonomously yet collaboratively to achieve the overall goal of planning a perfect vacation.

Structuring tasks for agents

Tasks define what an agent needs to accomplish. Each task is associated with an agent and includes a detailed description, expected output, and other relevant details.

The importance of structuring tasks

Structuring tasks is a critical step in the workflow of Crew AI. Tasks provide a clear roadmap for the agents, specifying what needs to be done, how to do it, and what the expected outcome should be.

Here’s why structuring tasks is so important:

1. Clarity: Tasks offer a clear and concise description of what is expected, helping agents focus on specific goals.
2. Guidance: They provide step-by-step instructions and necessary parameters, ensuring agents have all the information they need to complete their tasks.
3. Consistency: Structured tasks ensure that agents produce consistent and reliable outputs, which are essential for integrating the results into the overall project.

How tasks work

Tasks define specific objectives for agents, detailing the work to be done, expected outputs, and any necessary inputs. Each task is tailored to the agent’s role and capabilities, specifying the format and structure of the output to standardize results.

For our automated vacation planner, tasks are structured as follows:

• Description: Provides a detailed explanation of what the agent needs to do.
• Expected Output: Specifies the format and type of output expected from the agent.
• Human Input: Indicates whether human input is required for the task.
• Output Format: Defines the structure of the output using a model.
• Associated Agent: Specifies the agent responsible for completing the task.

Example task for the Venue Coordinator

We define and structure a task for the Venue Coordinator agent:

from crewai import Task
from pydantic import BaseModel
from datetime import date

# Define a Pydantic model for venue details
class RentalDetails(BaseModel):
   name: str
   address: str
   capacity: int
   price_per_night: int
   start_date: date
   end_date: date
   description: str
   amenities: list[str]
   source: str

# Define the task for the Venue Coordinator agent
hotel_task = Task(
   description="Find a hotel or rental in {vacation_city} that meets criteria for {vacation_details}, {budget}, {group_size}, {start_date}, and {end_date}.",
   expected_output="Details of suitable rental options, including name, address, capacity, price per night, available dates, description, and amenities.",
   human_input=True,
   output_json=RentalDetails,
   output_file="venue_details.json",
   agent=rental_coordinator
)

Understanding the Code

• Pydantic Model: We use Pydantic to define a model for the output data. The RentalDetails class specifies the structure of the rental details, ensuring consistency and ease of use.
• Task Definition: The hotel_task variable defines a task for the Rental Coordinator agent. It outlines the task's objective, expected output, need for human input, and links the task to the agent.

Understanding how agents and tasks use tools

In our automated vacation planner, agents use these tools to perform their designated tasks. For example:

• Rental Coordinator: Uses the search_tool to find suitable accommodation options and the scrape_tool to extract detailed information from relevant websites.
• Itinerary Planner: Uses the same tools to search for and gather information about activities and excursions.
• Budgeting Agent: Uses the search_tool to find cost-related information and ensure the vacation plans stay within budget.

Here’s a brief look at how an agent uses a tool within a task:

# Define the Venue Coordinator agent
rental_coordinator = Agent(
   role="Hotel/Rental Coordinator",
   goal="Identify an appropriate hotel, rental home, or vacation rental.",
   tools=[search_tool, scrape_tool],
   verbose=True,
   backstory=(
     "With a keen sense of space and understanding of vacation logistics, "
     "you excel at finding and securing the perfect vacation rental that fits "
     "the vacation's location, start and end dates, group size, and budget constraints."
  )
)

# Example task using the tools
hotel_task = Task(
   description="Find a hotel or rental in {vacation_city} that meets criteria for {vacation_details}, {budget}, {group_size}, {start_date}, and {end_date}.",
   expected_output="Details of suitable rental options, including name, address, capacity, price per night, available dates, description, and amenities.",
   human_input=True,
   output_json=RentalDetails,
   output_file="venue_details.json",
   agent=rental_coordinator
)

In this code, the rental_coordinator agent uses the search_tool and scrape_tool to find and extract information about potential accommodations. The hotel_task defines what the agent needs to accomplish, and the tools provide the necessary functionalities to complete the task.

By leveraging these tools, our agents can efficiently gather and process the information needed to plan a perfect vacation. Tools enable agents to perform their tasks effectively, ensuring that the entire planning process is streamlined and accurate.

Managing the Crew

The crew is a collection of agents working together to achieve the overall goal.

In Crew AI, managing the crew involves defining a group of agents, assigning tasks to them, and orchestrating their activities to ensure they work in harmony. This coordination is crucial for achieving a cohesive and efficient workflow.

The role of the Crew

The crew acts as the central unit that brings together various agents and tasks. It ensures that each agent performs its role effectively and that the outputs of individual tasks are integrated smoothly into the overall project.

Here’s why managing the crew is important:

1. Coordination: The crew ensures that all agents work together in a coordinated manner, avoiding conflicts and overlaps.
2. Efficiency: By managing the process flow, the crew optimizes the execution of tasks, saving time and resources.
3. Scalability: The crew allows for the addition of more agents and tasks as needed, making the system scalable for larger and more complex projects.

How to manage the Crew

Managing the crew involves several steps: defining the agents, assigning tasks, and orchestrating the process flow using a language model (LLM).

Here’s a detailed explanation of how we manage the crew in our vacation planning project.

Defining the Crew:

1. Agents: We start by defining the agents that will be part of the crew. Each agent has a specific role and set of tasks.
2. Tasks: We assign tasks to the agents, ensuring each task is aligned with the agent’s capabilities and goals.
3. Process Flow: We define the process flow, specifying how tasks are executed and how agents interact.

Example of managing the crew

Let’s look at an example of how we define and manage the crew for our vacation planning project:

from crewai import Crew, Process
from langchain_openai import ChatOpenAI

# Example data for kicking off the process
vacation_details = {
   'vacation_city': "Honolulu",
   'vacation_details': "A vacation for an adventurous family of 7 who want to explore the island, see the nature, and experience some good Hawaiian food and culture",
   'start_date': "2024-06-15",
   'end_date': "2024-06-22",
   'group_size': 7,
   'budget': 10000,
}

# Define the crew with the venue coordinator agent, itinerary planner agent, and budgeting agent
vacation_planning_crew = Crew(
   agents=[rental_coordinator, itinerary_planner, budgeting_agent],
   tasks=[hotel_task, itinerary_task, budgeting_task],
   manager_llm=ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7),
   process=Process.hierarchical,
   verbose=True
)

Understanding the code

Defining agents and tasks:

• We include our previously defined agents (rental_coordinator, itinerary_planner, budgeting_agent) and their respective tasks (hotel_task, itinerary_task, budgeting_task).

Example data:

• The vacation_details dictionary contains example data that provides context for the tasks. This includes the destination city, details about the vacation, dates, group size, and budget.

Initializing the crew:

• The crew class is used to define the crew. We pass the list of agents and tasks to the crew constructor.
• The manager_llm parameter specifies the language model used to manage the crew. In this case, we use ChatOpenAI with the model gpt-3.5-turbo.
• The process parameter is set to Process.hierarchical, indicating that the tasks will be managed in a hierarchical manner, ensuring a logical sequence of execution.
• The verbose parameter is set to True to enable detailed logging, which helps in monitoring the process flow and debugging if needed.

How the crew operates

Once the crew is defined, it operates by coordinating the activities of the agents:

• Task Execution: The crew ensures that each task is executed in the correct order. For example, the Venue Coordinator agent first finds suitable accommodation options, which are then used by the Itinerary Planner agent to plan daily activities.
• Data Integration: The output of one task can be used as input for another. For instance, the details of the selected accommodation are used to plan activities that are conveniently located.
• Management and Monitoring: The language model (LLM) manages the overall process, making decisions on task execution, and ensuring that agents collaborate effectively. The hierarchical process ensures that tasks are performed in a logical sequence, avoiding conflicts, and ensuring smooth integration.

By managing the crew in this structured manner, we ensure that all agents work together seamlessly to achieve the overall goal of planning a perfect vacation.

This approach can be extended to other complex projects, providing a scalable and efficient solution for multi-agent collaboration.

To Review…

By using AI agents, tools, tasks, and a crew, we’ve automated the process of planning a vacation. This approach can be extended to other use cases, such as event planning, customer support, and more.

This tutorial has shown how to create a multi-agent system that collaborates to achieve a common goal. Understanding how these components work together is key to leveraging the power of AI in automation tasks.

Supporting Code on GitHub

You can find the supporting complete code in the GitHub repository.

This demonstrates the processes outlined above for creating a multi-agent vacation planner using crewAI. This includes:

• The travel_agent.py script
• A requirements.txt file
• A ReadME.md file to guide you through this process
• The utils.py script with utility functions for fetching API keys

Additional Examples with Multiple Agents:

• The python_writer script: Describe a task you want implemented in Python and the crew will output the Python code.
• The stock_analysis.py script: Describe a stock you want analyzed in the market and the crew will give feedback on it.
• The resume_builder.py script: Give the crew your current resume in markdown format along with a job posting link and your github. It will tailor the resume to the job you are applying for. The fake_resume.mdfile is included in the repo for you to play around with the code.