Designing effective AI agent systems involves using various design patterns to ensure efficient, robust, and maintainable workflows. In previous post, we introduced single, multiple agents and sequence chats. In this post, we will introduce two key design patterns: reflection using a critic and nested agents. These patterns enable more sophisticated interactions and workflows, enhancing the capabilities of AI-driven systems.
AutoGen supports different LLM prompting and reasoning strategies, such as ReAct, Reflection/Self-Critique, and more.
Reflection is a pattern where agents evaluate their actions or decisions with the help of a critic agent. The critic, typically an AssistantAgent, provides feedback, ensuring that the main agent’s responses are accurate and appropriate. This pattern is useful for scenarios requiring high accuracy and quality control, such as content generation, decision-making, and troubleshooting.
Reflection Using a Critic (AssistantAgent)
Reflection involves an agent performing a task and then sending the result to a critic agent. The critic evaluates the result and provides feedback, which the original agent uses to improve or validate its response.
from autogen import ConversableAgent,AssistantAgent, UserProxyAgent
from autogen import config_list_from_json
if __name__ == "__main__":
# setup config_list
# if using OAI_CONFIG_LIST_LOCAL: llama3 served by LM Studio at local.
config_list = config_list_from_json(env_or_file="../utils/OAI_CONFIG_LIST_LOCAL")
llm_config = {
"seed": 44, # Set seed for reproducibility
"config_list": config_list, # Pass configuration list to LLM
"temperature": 0 # Set temperature for model generation
}
content_generation_agent = AssistantAgent(
name="Content Generation Agent",
system_message='''You are a content generation assistant.
Your job is to create content based on user input.
Ensure the content is accurate and well-structured.
Send your output to the Critic Agent for review.''',
llm_config=llm_config,
code_execution_config=False,
human_input_mode="NEVER",
)
# Define the Critic Agent
critic_agent = AssistantAgent(
name="Critic Agent",
system_message='''You are a critic assistant.
Your job is to review the content generated by the Content Generation Agent.
Provide feedback on accuracy, structure, and quality.
Suggest improvements if necessary. Return 'APPROVED' if the content is satisfactory.''',
llm_config=llm_config,
code_execution_config=False,
human_input_mode="NEVER",
is_termination_msg=lambda x: x.get("content", "").find("TERMINATE") >= 0,
)
task = "Write an introduction about AI in healthcare."
res = critic_agent.initiate_chat(
recipient=content_generation_agent,
message=task,
max_turns=2,
summary_method="last_msg"
)
Nested Agent for Reflection
Nested Agents involve creating a hierarchical structure where an agent manages one or more subordinate agents to perform complex tasks. This pattern is useful for breaking down large tasks into manageable sub-tasks, enabling modular and scalable workflows.
Nested agents follow a top-down approach where a primary agent delegates specific tasks to subordinate agents. These subordinate agents perform their tasks and report back to the primary agent, which consolidates the results.
Multi-Step Customer Support Example
Primary Support Agent: Manages the overall support process and delegates tasks to specialized agents.
Subordinate Agents: Handle specific parts of the support process, such as account issues, technical troubleshooting, and billing inquiries.
Here is one example about nested agents, but there are two (symptom_collection_agent and medical_history_agent) inside agents still need to get info from User by human-interactions
config_list = config_list_from_json(env_or_file="../utils/OAI_CONFIG_LIST")
llm_config = {
"model": "gpt-3.5-turbo",
"seed": 44,
"config_list": config_list,
"temperature": 0
}
# Define the agents
# Allow user interaction if this agent need some answers/info from user
symptom_collection_agent = ConversableAgent(
name="Symptom Collection Agent",
system_message='''You are responsible for collecting detailed symptom information from the patient.
Ask relevant questions and ensure all symptoms are recorded accurately.
Return 'TERMINATE' when you have gathered all the information.''',
llm_config=llm_config,
code_execution_config=False,
human_input_mode="ALWAYS",
)
medical_history_agent = ConversableAgent(
name="Medical History Agent",
system_message='''You are responsible for gathering and reviewing the patient's medical history.
Ask about past illnesses, surgeries, and any relevant medical information.
Return 'TERMINATE' when you have gathered all the information.''',
llm_config=llm_config,
code_execution_config=False,
human_input_mode="ALWAYS",
)
diagnostic_analysis_agent = AssistantAgent(
name="Diagnostic Analysis Agent",
system_message='''You are responsible for analyzing the collected data to suggest potential diagnoses.
Use the information from the symptom collection and medical history to make informed suggestions.
Return 'TERMINATE' when you have completed the analysis.''',
llm_config=llm_config,
code_execution_config=False,
human_input_mode="NEVER",
)
treatment_recommendation_agent = AssistantAgent(
name="Treatment Recommendation Agent",
system_message='''You are responsible for providing treatment options based on the diagnosis.
Ensure the recommendations are clear and actionable for the patient.
Return 'TERMINATE' when you have provided all the necessary recommendations.''',
llm_config=llm_config,
code_execution_config=False,
human_input_mode="NEVER",
)
# Primary diagnostic agent coordinating the nested agents
primary_diagnostic_agent = AssistantAgent(
name="Primary Diagnostic Agent",
system_message='''You are responsible for coordinating the diagnostic workflow.
Ensure that the symptom collection, medical history, diagnostic analysis,
and treatment recommendation agents complete their tasks in sequence.''',
llm_config=llm_config,
code_execution_config=False,
human_input_mode="NEVER",
)
user_proxy = UserProxyAgent(
name="User",
human_input_mode="ALWAYS",
is_termination_msg=lambda x: x.get("content", "").find("TERMINATE") >= 0,
code_execution_config={
"last_n_messages": 1,
"work_dir": "tasks",
"use_docker": False,
},
)
# Define the workflow for nested agents by sorted orders
chats = [
{
"recipient": symptom_collection_agent,
"message": "Please start collecting the patient's symptoms.",
"summary_method": "reflection_with_llm",
"summary_args": {
"summary_prompt": "Summarize the collected symptoms as JSON: {'symptoms': []}",
},
"max_turns": 1,
"clear_history": True
},
{
"recipient": medical_history_agent,
"message": "Please gather the patient's medical history.",
"summary_method": "reflection_with_llm",
"summary_args": {
"summary_prompt": "Summarize the medical history as JSON: {'medical_history': []}",
},
"max_turns": 1,
"clear_history": True
},
{
"recipient": diagnostic_analysis_agent,
"message": "Please analyze the collected data and suggest potential diagnoses.",
"summary_method": "reflection_with_llm",
"summary_args": {
"summary_prompt": "Summarize the diagnostic analysis as JSON: {'diagnosis': []}",
},
"max_turns": 1,
"clear_history": True
},
{
"recipient": treatment_recommendation_agent,
"message": "Please provide treatment options based on the diagnosis.",
"summary_method": "reflection_with_llm",
"summary_args": {
"summary_prompt": "Summarize the treatment recommendations as JSON: {'treatments': []}",
},
"max_turns": 1,
"clear_history": True
}
]
def reflection_message(recipient, messages, sender, config):
print("Reflecting...", "now")
return f"Review the following content. \n\n {recipient.chat_messages_for_summary(sender)[-1]['content']}"
# Initiate the nested agent chats after get user's 1st response
primary_diagnostic_agent.register_nested_chats(chats, trigger=user_proxy)
# Start the interaction by primary_diagnostic_agent
# primary_diagnostic_agent is aksing User.
querytask = "What brings you in today?"
chat_results = primary_diagnostic_agent.initiate_chat(
recipient=user_proxy,
message=querytask,
max_turns=2,
summary_method="last_msg"
)
pprint.pprint("-----------------------END----------------------")
+--------------------------+
| Primary Diagnostic Agent|
+--------------------------+
/|\
|
|
+-----------------+-------------------+
| | |
| | |
| | |
| | |
| | |
| | |
+--------------------------+ +-------------------------+ +-------------------------+ +-------------------------+
| Symptom Collection Agent | | Medical History Agent | | Diagnostic Analysis Agent| |Treatment Recommendation |
| (Interacts with User) | | (Interacts with User) | | | | Agent |
+--------------------------+ +-------------------------+ +-------------------------+ +-------------------------+
| /|\ /|\ /|\
| | | |
| | | |
+-----------------+-------------------------+--------------------------+
|
|
+--------------------+
| User Proxy |
| (Interacts with User|
+--------------------+Agents relationship:
Primary Diagnostic Agent at the center coordinates the entire process.
User Proxy interacts with the user and serves as the trigger for the nested workflow.
Four arrows extend from the Primary Diagnostic Agent to the other agents:
Symptom Collection Agent, Medical History Agent, Diagnostic Analysis Agent, Treatment Recommendation Agent.
The sequence of interactions:
Symptom Collection Agent and Medical History Agent both have bidirectional arrows with the User Proxy, indicating direct user interaction.
Diagnostic Analysis Agent and Treatment Recommendation Agent have unidirectional arrows from the Primary Diagnostic Agent, indicating they receive information and provide output without direct user interaction.
Conclusion
Reflection is a general prompting strategy which involves having LLMs analyze their own outputs, behaviors, knowledge, or reasoning processes. Reflection ensures high-quality outputs through critical evaluation, while nested agents enable modular and scalable task management.
Here, this preliminary example is using LLM gpt-3.5-turbo from OpenAI, also we could try by using local open source ones, like llama3 or others.
References:
AI Agentic Design Patterns with AutoGen
AutoGen LLM Reflection
