“Web of Agents”: Elevating GenAI with Advanced Multi-Agent Frameworks for Scalable Deployments
In this article, I would like to offer a vision and specification of how GenAI multi-agent environments can scale at enterprise level. I will introduce the new concept of “Web of Agents”, a distributed and decentralized network of GenAI agents that can interact and cooperate with each other and with the users through a common interface and protocol.
Intro: From prompt engineering to Gen AI-enabled agents
Generative AI (Gen AI) is a revolutionary technology that enables natural language understanding and generation across multiple domains and languages. GenAI applications, such as chatbots, assistants, translators, or summarizers, can leverage the power of large-scale pre-trained language models to provide high-quality and diverse responses to user queries or commands. However, most of the current GenAI applications are based on prompt engineering and RAG (retrieval-augmented generation), which have some limitations in terms of scalability, flexibility, and robustness.
To overcome these challenges, there is upcoming trend of GenAI-enabled multi-agent environments, where multiple agents, each with their own GenAI capabilities, can collaborate and communicate with each other and with the users to achieve complex goals. For example, a travel booking agent can interact with a flight search agent, a hotel booking agent, and a weather forecast agent to plan a trip for a user. Each agent can specialize in a specific domain or task and use its own GenAI model to generate natural language responses. The agents can also exchange information and requests to each other by using a common protocol or language. This way, the multi-agent environment can leverage the strengths of each individual agent and provide a more comprehensive and coherent service to the user.
Challenge
However, GenAI-enabled multi-agent environments also pose some new challenges and questions, such as:
· How to ensure that the agents are compatible and interoperable with each other, regardless of their underlying GenAI models, platforms, or architectures?
· How to coordinate and orchestrate the interactions among the agents and the users, especially when there are conflicts, uncertainties, or ambiguities in the communication or the goal?
· How to monitor and evaluate the performance and behavior of the agents, both individually and collectively, and ensure that they adhere to ethical and quality standards?
· How to scale and maintain the multi-agent environment as the number and diversity of the agents and the users grow?
New concept: What is a “Web of Agents” ?
Web of Agents (WOA) refers to a distributed and decentralized network of GenAI agents that can interact and cooperate with each other and with the users through a common interface and protocol. The WOA architecture facilitates the integration and cooperation of multiple GenAI agent teams, enabling them to address complex challenges more efficiently through collective intelligence and capabilities.
“Web of Agents” components
Gen AI Multi-Agent Team: Refers to a collection of GenAI-enabled agents that are coordinated by a chat manager. These teams are specifically constructed to tackle domain-specific or use-case-specific challenges.
GenAI-Enabled Agents: Refers to individual software components that leverage Gen AI to autonomously address particular problems, either independently or in collaboration with other agents. More details about the ‘anatomy’ of agents can be found in my other article.
Chat Manager: A sophisticated component that varies in complexity and is crucial for the specific implementation of the multi-agent system. It orchestrates task execution, manages communication between agents via chat messages, and assesses the team’s performance using defined metrics.
Orchestration problem
Based on my experience working with agents, the primary challenge lies in orchestration. Deploying more than five agents often results in the chat manager mistakenly assigning tasks to the wrong agent. Additionally, in some frameworks, agents are permitted to assign tasks to other agents. However, they may not be appropriately configured for these tasks, leading to ambiguity in task assignments. Consequently, the more agents involved, the greater the potential for disorganization.
Mastering agent orchestration at scale
I would like to propose a scalable solution for agent orchestration with the following rules:
Rule 1: Specialized Chat Managers
Introducing the threetypes of Chat Managers:
- Internal to Multi-Agent Team (Level 1): Orchestrates activities within a single team
- Between Two or More Gen AI Multi-Agent Teams Within the Organization (Level 2): Facilitates coordination and collaboration between multiple internal teams
- Between Two or More Gen AI Multi-Agent Teams, Including External Teams (Level 3): Manages interactions and integrations between internal teams and at least one external team
The expectation is that even the most complex problems should not require more than five GenAI Multi-Agent Teams. Even if this number is exceeded, the task at Level 3 orchestration should be pretty straightforward for unambiguous assignment.
Rule 2: Independence and Prioritization
Each instance of the orchestration mechanism operates independently from others, ensuring tailored and efficient management of tasks and communications. In scenarios where multiple levels of orchestration are active simultaneously, the priority is given in descending order from Level 3 (highest) to Level 1 (lowest), reflecting the increasing complexity and scope of coordination required at higher levels.
Rule 3: Unambiguous Assignments
The Chat Manager may assign tasks to an Agent or another Chat Manager. Agents are not permitted to assign tasks.
How it works: An illustrative example
Request
Analyze bank’s retail products and create enhancement opportunities based on market projections and bank’s data.
WOA architecture
The ‘Web of Agents’ solution design for this request comprises four Gen AI Multi-Agent teams (three internal and one external):
- Team 1 (Internal): Market Projection Analysis — Provides and analyzes data from external sources.
- Team 2 (External): Market Data Provider — Provides market projection data to Team 1.
- Team 3 (Internal): Bank’s Data Analysis — Provides and analyzes the bank’s product data.
- Team 4 (Internal): Final Answer Agent — Summarizes results from Team1 and Team2 and creates PowerPoint presentations.
Chat messages (from execution)
Below are chat messages from a hypothetical execution.
With this approach, you may experience slightly more cycles, but the agent communication is ‘clean’ and unambiguous, and the request is fulfilled effectively.
Conclusion
I hope that this article will inspire and stimulate further research and development of “Web of Agents”, GenAI multi-agent environments at scale and contribute to the advancement and adoption of GenAI technology in the real world. I welcome any feedback, comments, or suggestions from the readers and the GenAI community.
Disclaimer: The views and opinions expressed in this article are those of the author and do not necessarily reflect the opinions or positions of any entities author represents.
