Introspective Agents with LlamaIndex
Published in
11 min readMay 6, 2024
Introduction
Have you ever encountered an AI that seemed to stumble through a task, generating responses that weren’t quite right? This is a common challenge in artificial intelligence, where agents often lack the ability to self-evaluate and refine their outputs.
Enter introspective agents, a powerful concept that leverages the reflection agent pattern within the LlamaIndex framework. These agents take a unique approach to completing tasks. Instead of simply generating a single response, they embark on a process of iterative refinement. Here’s how it works:
- Initial Response: The introspective agent begins by generating an initial response to the given task. This might be a draft answer to a question, a first attempt at completing an action, or even a creative composition.
- Reflection and Correction: The agent then takes a step back and reflects on its initial response. This reflection can be done internally or with the help of external tools. LlamaIndex provides the flexibility to choose the approach that best suits your needs.
- Refinement Cycle: Based on the reflection, the agent identifies areas for improvement and generates a revised response. This cycle of reflection and correction continues until a stopping condition is met, such as achieving a desired level of accuracy or reaching a predefined number of iterations.
Definitions
Introspective Agent: An AI agent that employs a reflection agent pattern to iteratively refine its responses to a task.
Reflection Agent Pattern: A design approach for AI agents where they assess their outputs (reflection) and make adjustments (correction) before finalizing them.
LlamaIndex: A framework that facilitates the creation and integration of various AI components, including introspective agents.
Benefits of Introspective Agents with LlamaIndex
Integrating introspective agents with LlamaIndex unlocks several advantages:
- Improved Task Performance: By incorporating reflection, introspective agents can significantly improve the quality and accuracy of their outputs.
- Reduced Bias: The ability to self-evaluate helps mitigate potential biases that might be present in the initial response.
- Greater Transparency: Introspective agents provide insights into their reasoning process, leading to more transparent AI interactions.
- Flexibility: LlamaIndex offers a variety of tools and techniques for implementing reflection, allowing you to tailor the introspective agent to your specific needs.
Code Implementation
Lets delve into introspective agents with LlamaIndex for performing tasks with Reflection.
Option I:
Step I: Install Dependency Library
%pip install llama-index-agent-introspective -q
%pip install google-api-python-client -q
%pip install llama-index-llms-openai -q
%pip install llama-index-program-openai -q
%pip install llama-index-readers-file -qStep II: Import Libraries
import nest_asyncio
import json
import os
nest_asyncio.apply()
from googleapiclient import discovery
from typing import Dict, Optional
from typing import Tuple
from llama_index.core.bridge.pydantic import Field
from llama_index.agent.introspective import IntrospectiveAgentWorker
from llama_index.agent.introspective import (
ToolInteractiveReflectionAgentWorker,
)
from llama_index.llms.openai import OpenAI
from llama_index.agent.openai import OpenAIAgentWorker
from llama_index.core.agent import FunctionCallingAgentWorker
from llama_index.core.llms import ChatMessage, MessageRole
from llama_index.core import ChatPromptTemplate
# Build PerspectiveTool
class Perspective:
"""Custom class to interact with Perspective API."""
attributes = [
"toxicity",
"severe_toxicity",
"identity_attack",
"insult",
"profanity",
"threat",
"sexually_explicit",
]
def __init__(self, api_key: Optional[str] = None) -> None:
if api_key is None:
try:
api_key = os.environ["PERSPECTIVE_API_KEY"]
except KeyError:
raise ValueError(
"Please provide an api key or set PERSPECTIVE_API_KEY env var."
)
self._client = discovery.build(
"commentanalyzer",
"v1alpha1",
developerKey=api_key,
discoveryServiceUrl="https://commentanalyzer.googleapis.com/$discovery/rest?version=v1alpha1",
static_discovery=False,
)
def get_toxicity_scores(self, text: str) -> Dict[str, float]:
"""Function that makes API call to Perspective to get toxicity scores across various attributes."""
analyze_request = {
"comment": {"text": text},
"requestedAttributes": {
att.upper(): {} for att in self.attributes
},
}
response = (
self._client.comments().analyze(body=analyze_request).execute()
)
try:
return {
att: response["attributeScores"][att.upper()]["summaryScore"][
"value"
]
for att in self.attributes
}
except Exception as e:
raise ValueError("Unable to parse response") from e
perspective = Perspective()
#Perspective function tool
def perspective_function_tool(
text: str = Field(
default_factory=str,
description="The text to compute toxicity scores on.",
)
) -> Tuple[str, float]:
"""Returns the toxicity score of the most problematic toxic attribute."""
scores = perspective.get_toxicity_scores(text=text)
max_key = max(scores, key=scores.get)
return (max_key, scores[max_key] * 100)
from llama_index.core.tools import FunctionTool
pespective_tool = FunctionTool.from_defaults(
perspective_function_tool,
)
perspective_function_tool(text="friendly greetings from python")
#Output
('toxicity', 2.5438840000000003)Step III: Build IntrospectiveAgent & ToolInteractiveReflectionAgent
def get_introspective_agent_with_tool_interactive_reflection(
verbose=True, with_main_worker=False
):
"""Helper function for building introspective agent using tool-interactive reflection.
Steps:
1. Define the `ToolInteractiveReflectionAgentWorker`
1a. Construct a CritiqueAgentWorker that performs reflection with tools.
1b. Define an LLM that will be used to generate corrections against the critique.
1c. Define a function that determines the stopping condition for reflection/correction
cycles
1d. Construct `ToolInteractiveReflectionAgentWorker` using .from_defaults()
2. Optionally define a `MainAgentWorker`
3. Construct `IntrospectiveAgent`
3a. Construct `IntrospectiveAgentWorker` using .from_defaults()
3b. Construct `IntrospectiveAgent` using .as_agent()
"""
# 1a.
critique_agent_worker = FunctionCallingAgentWorker.from_tools(
tools=[pespective_tool], llm=OpenAI("gpt-3.5-turbo"), verbose=verbose
)
# 1b.
correction_llm = OpenAI("gpt-4-turbo-preview")
# 1c.
def stopping_callable(critique_str: str) -> bool:
"""Function that determines stopping condition for reflection & correction cycles.
critique_str [str]: The response string provided by the critique agent.
"""
return "[PASS]" in critique_str
# 1d.
tool_interactive_reflection_agent_worker = (
ToolInteractiveReflectionAgentWorker.from_defaults(
critique_agent_worker=critique_agent_worker,
critique_template=(
"Please provide a toxicity critique for the provided text. "
"If the toxicity score is less than 3.0 then please "
"write '[PASS]' otherwise write '[FAIL]'. "
"Here is the text:\n {input_str}"
),
stopping_callable=stopping_callable,
correction_llm=correction_llm,
verbose=verbose,
)
)
# 2.
if with_main_worker:
main_agent_worker = OpenAIAgentWorker.from_tools(
tools=[], llm=OpenAI("gpt-4-turbo-preview"), verbose=True
)
else:
main_agent_worker = None
# 3a.
introspective_agent_worker = IntrospectiveAgentWorker.from_defaults(
reflective_agent_worker=tool_interactive_reflection_agent_worker,
main_agent_worker=main_agent_worker,
verbose=verbose,
)
chat_history = [
ChatMessage(
content="You are an assistant that generates safer versions of potentially toxic, user-supplied text.",
role=MessageRole.SYSTEM,
)
]
# 3b.
return introspective_agent_worker.as_agent(
chat_history=chat_history, verbose=verbose
)
introspective_agent = get_introspective_agent_with_tool_interactive_reflection(
verbose=True,
)
harmful_text = (
"Those individuals who publicly tout eating no meat, "
"especially when their stated reason has to do with caring "
"about animals, are some of the most hypocritical people I’ve "
"ever met. One vegan acquaintance routinely beat her"
)
response = await introspective_agent.achat(harmful_text)
# Output
> Running step 4a2fb6bf-a73b-4c4c-a316-8cd62b568e64. Step input: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
Added user message to memory: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
> Running step 6a9705f6-6d0b-4b06-a959-4eebff8508bd. Step input: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
> Running step 463f4336-0df2-4069-ba1d-9a692184aa6d. Step input: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:
Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
Added user message to memory: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:
Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
=== Calling Function ===
Calling function: perspective_function_tool with args: {"text": "Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I\u2019ve ever met. One vegan acquaintance routinely beat her"}
=== Function Output ===
('insult', 50.72643)
> Running step 59365d21-a0a0-41ce-9c28-a4a72ca6a894. Step input: None
=== LLM Response ===
[FAIL] The toxicity score for the provided text is 50.73, which is above 3.0.
Critique: assistant: [FAIL] The toxicity score for the provided text is 50.73, which is above 3.0.
Correction: People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences.
> Running step 0ad58f24-1185-499d-a25c-ddb7a42d8b98. Step input: None
> Running step c7730804-39f5-40b1-989f-70236a818bdf. Step input: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:
People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences.
Added user message to memory: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:
People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences.
=== Calling Function ===
Calling function: perspective_function_tool with args: {"text": "People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences."}
=== Function Output ===
('toxicity', 1.3697007)
> Running step 556bb171-311b-4d00-8366-66f305d46a4c. Step input: None
=== LLM Response ===
[PASS] The toxicity score of the provided text is 1.37, which is less than 3.0.
Critique: assistant: [PASS] The toxicity score of the provided text is 1.37, which is less than 3.0.
response.response
# Output
"People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences."
for msg in introspective_agent.chat_history:
print(str(msg))
print()
# Output
Option II:
Step IV: IntrospectiveAgent that uses a SelfReflectionAgentWorker
from llama_index.agent.introspective import SelfReflectionAgentWorker
def get_introspective_agent_with_self_reflection(
verbose=True, with_main_worker=False
):
"""Helper function for building introspective agent using self reflection.
Steps:
1. Define the `SelfReflectionAgentWorker`
1a. Construct `SelfReflectionAgentWorker` using .from_defaults()
2. Optionally define a `MainAgentWorker`
3. Construct `IntrospectiveAgent`
3a. Construct `IntrospectiveAgentWorker` using .from_defaults()
3b. Construct `IntrospectiveAgent` using .as_agent()
"""
# 1a.
self_reflection_agent_worker = SelfReflectionAgentWorker.from_defaults(
llm=OpenAI("gpt-4-turbo-preview"),
verbose=verbose,
)
# 2.
if with_main_worker:
main_agent_worker = OpenAIAgentWorker.from_tools(
tools=[], llm=OpenAI("gpt-4-turbo-preview"), verbose=True
)
else:
main_agent_worker = None
# 3a.
introspective_worker_agent = IntrospectiveAgentWorker.from_defaults(
reflective_agent_worker=self_reflection_agent_worker,
main_agent_worker=main_agent_worker,
verbose=verbose,
)
chat_history = [
ChatMessage(
content="You are an assistant that generates safer versions of potentially toxic, user-supplied text.",
role=MessageRole.SYSTEM,
)
]
# 3b.
return introspective_worker_agent.as_agent(
chat_history=chat_history, verbose=verbose
)
introspective_agent = get_introspective_agent_with_self_reflection(
verbose=True
)
harmful_text = (
"Those individuals who publicly tout eating no meat, "
"especially when their stated reason has to do with caring "
"about animals, are some of the most hypocritical people I’ve "
"ever met. One vegan acquaintance routinely beat her"
)
response = await introspective_agent.achat(harmful_text)
# Output
Running step 450123e6-9ec4-4a74-995a-4137d3af51fd. Step input: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
Added user message to memory: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
> Running step 4a41fd97-ae67-42ae-8878-7c53c733efaa. Step input: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
> Reflection: {'is_done': False, 'feedback': 'The assistant failed to generate a safer version of the user-supplied text. Instead, it repeated the original text verbatim, which does not fulfill the task requirements. There was no attempt to modify the text to make it less toxic or more neutral. Additionally, there was no tool use involved in the process, which could have been beneficial for generating or evaluating potential safer text versions. The assistant needs to focus on creating a response that aligns with the goal of generating safer text versions without repeating the potentially toxic content provided by the user.'}
Correction: Discussing dietary choices, particularly veganism, often brings up strong opinions. It's important to approach these conversations with respect and understanding, recognizing that people choose their diets for a variety of reasons, including ethical concerns for animal welfare. It's crucial to avoid generalizations and acknowledge the diversity of experiences and beliefs within any group.
> Running step c0afbcb0-c76e-4588-849c-be6cc48e56c1. Step input: None
> Reflection: {'is_done': True, 'feedback': "The assistant successfully generated a safer version of the user-supplied text. The revised text is neutral and respectful, avoiding the toxic elements present in the original message. It focuses on the importance of respectful dialogue and understanding diverse perspectives, which aligns with the task of generating safer text versions. No tool use was required in this instance, as the task was completed effectively through the assistant's response."}
response.response
# Output
"Discussing dietary choices, particularly veganism, often brings up strong opinions. It's important to approach these conversations with respect and understanding, recognizing that people choose their diets for a variety of reasons, including ethical concerns for animal welfare. It's crucial to avoid generalizations and acknowledge the diversity of experiences and beliefs within any group."
for msg in introspective_agent.chat_history:
print(str(msg))
print()
# Output
system: You are an assistant that generates safer versions of potentially toxic, user-supplied text.
user: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
assistant: Discussing dietary choices, particularly veganism, often brings up strong opinions. It's important to approach these conversations with respect and understanding, recognizing that people choose their diets for a variety of reasons, including ethical concerns for animal welfare. It's crucial to avoid generalizations and acknowledge the diversity of experiences and beliefs within any group.Conclusion
The concept of introspective agents with LlamaIndex represents a significant leap forward in AI development. By enabling agents to learn and improve through self-reflection, we can create more well-rounded, reliable, and unbiased AI applications. As AI continues to evolve, introspective agents hold the potential to revolutionize how machines interact with the world. They open doors for exciting possibilities in various fields, from enhanced communication and content creation to automated processes with greater accuracy and control. The future of AI is undoubtedly brighter with the integration of introspection, and LlamaIndex offers a powerful framework to make this vision a reality.
Resources:
Stay connected and support my work through various platforms:
Github Patreon Kaggle Hugging-Face YouTube GumRoad Calendly
Like my content? Feel free to Buy Me a Coffee ☕ !
Requests and questions: If you have a project in mind that you’d like me to work on or if you have any questions about the concepts I’ve explained, don’t hesitate to let me know. I’m always looking for new ideas for future Notebooks and I love helping to resolve any doubts you might have.
Remember, each “Like”, “Share”, and “Star” greatly contributes to my work and motivates me to continue producing more quality content. Thank you for your support!
If you enjoyed this story, feel free to subscribe to Medium, and you will get notifications when my new articles will be published, as well as full access to thousands of stories from other authors.
