Using Agents as Tools in LangChain
--
If you’ve just started looking into LangChain and wonder how you could use agents as tools for other agents, you’ve come to the right place. This article quickly goes over the basics of agents in LangChain and goes on to a couple of examples of how you could make a LangChain agent use other agents.
LangChain has been emerging as the main OS library for building LLM applications of all kinds. One of the things that make it so powerful is the possibility to create agents. If you’ve come across BabyAGI or AutoGPT, you’ve encountered or even used these yourself — the graph below illustrates how BabyAGI is basically nothing else but various agents interacting with each other.
One of the first questions I had when I started to play around with LangChain was: How could you build a tree of agents, where one main agent chooses which of its subordinate agents is best suited for a given task?
The Setup
Most articles on LangChain you will find are using math or simple text use cases — we are trying out a slightly different example to maximize here, to maximize your and our learnings.
The idea is as follows: Given a price database, how could I calculate the total value of items mentioned in any given text? An example text could look like the following:
STORY = """In the heart of the apartment lies a room of quiet reflection and muted hues, whispering tales of homeliness
and tranquility. The floors, washed in the warmth of polished oak, play host to an eclectic collection of cherished
objects, each a beacon of the room's characteristic charm.
An emerald green sofa, its plush cushions like rolling hills of velvety softness, commands attention as it proudly sits
in the center of the room. It is a monument of many an evening spent immersed in the pages of a gripping novel or in
the laughter of shared stories. At its feet, a rug of rich burgundy, its fibers gently tickling bare feet, maps out the
heart of this sanctuary.
To its right, a humble coffee table of natural wood takes center stage. On it, a small collection of coffee-stained mugs
and well-thumbed novels narrate tales of quiet mornings and lazy afternoons. Above it, a lamp, its shade the color of
fresh cream, casts a warm, inviting glow across the room, shadows dancing in its light.
Against one wall stands a bookshelf, an aged sentinel of oak and iron. It houses a world of adventures and knowledge,
its shelves groaning under the weight of tomes old and new. Beside it, a chair, upholstered in soft, sky-blue fabric,
invites the weary to rest.
In the corner, a lone desk of polished mahogany stands. A testament to late-night musings and feverish scribblings,
its surface is littered with sheets of paper and an array of writing tools. Above it, a clock, its hands dancing in
an endless waltz, keeps rhythm with the heartbeat of the room.
Every item, every piece of furniture, is a verse in the poem that is this room. They paint a portrait of serenity,
a world within a world, where each moment unfurls with the soft rustle of turning pages and the comforting whispers
of home."""Next, we set up a simple json containing prices of various household items. We define an additional parameter volume to increase the difficulty for our agent a little
{
"sofa": {
"price": 1000.0,
"volume": 2.5
},
"television": {
"price": 500.0,
"volume": 0.02
},
"refrigerator": {
"price": 1200.0,
"volume": 0.8
},
"microwave": {
"price": 150.0,
"volume": 0.03
},
"bed": {
"price": 800.0,
"volume": 1.5
},
"desk": {
"price": 200.0,
"volume": 0.4
},
"chair": {
"price": 50.0,
"volume": 0.2
},
"lamp": {
"price": 30.0,
"volume": 0.01
},
"bookshelf": {
"price": 100.0,
"volume": 0.5
},
"computer": {
"price": 1200.0,
"volume": 0.02
},
"coffee table": {
"price": 75.0,
"volume": 0.2
},
"dining table": {
"price": 250.0,
"volume": 1.0
},
"wardrobe": {
"price": 350.0,
"volume": 1.5
},
"oven": {
"price": 500.0,
"volume": 0.3
},
"dishwasher": {
"price": 400.0,
"volume": 0.3
},
"washing machine": {
"price": 600.0,
"volume": 0.6
},
"toaster": {
"price": 20.0,
"volume": 0.01
},
"blender": {
"price": 30.0,
"volume": 0.02
},
"rug": {
"price": 80.0,
"volume": 0.1
},
"clock": {
"price": 25.0,
"volume": 0.01
}
}The first thing we define is a simple sum calculator tool. This one is not an agent tool yet. Note that as always, it’s important to give the function your tool uses detailed and verbose docstrings — this is additional info for your agent to decide whether this tool is appropriate to use at any given point
from langchain.tools import StructuredTool
def inventory_value_calculator(inventory_item_prices: list) -> float:
"""
This function helps you calculate the sum of prices of multiple inventories given a list of prices.
:param inventory_item_prices: List of inventory prices.
:return: The sum of prices
"""
room_inventory_value = sum(inventory_item_prices)
return room_inventory_value
def inventory_value_calculator_tool():
structured_tool = StructuredTool.from_function(
name="calculate the value of the inventory in a room.",
func=inventory_value_calculator,
description="Use this tool if you want to calculate the total value of the inventory of a room.")
return structured_toolNext, we define the agent that is supposed to be used as tool by another agent. Since we are dealing with reading from a JSON, I used the already defined json agent from the langchain library:
from langchain.agents import create_json_agent
from langchain.agents.agent_toolkits import JsonToolkit
from langchain.chat_models import ChatOpenAI
from langchain.tools.json.tool import JsonSpec
def get_json_agent(json_path: str):
with open(json_path) as f:
data = json.load(f)
json_spec = JsonSpec(dict_=data, max_value_length=4000)
json_toolkit = JsonToolkit(spec=json_spec)
json_agent = create_json_agent(
llm=ChatOpenAI(temperature=0),
toolkit=json_toolkit,
verbose=True
)
return json_agentI personally tried two different approaches and want to show what didn’t work for me as well.
Attempt 1 — Trying to create a tool from the agent.run method directly
The docs suggest using the agent.run method as function parameter to the StructuredTool.from_function method, which I tried first
from langchain.tools import StructuredTool
def agent_executor_as_tool():
json_agent = get_json_agent("./inventory_prices_dict.json")
json_tool = StructuredTool.from_function(
func=json_agent.run,
name="reading inventory prices",
description="useful for getting the price of a single inventory item or furniture element.",
)
return json_toolSince StructuredTool uses pydantic to translate data types and docstrings of the func parameter into JSON (to provide that to the using agent as extra info), we are running into an error here:
ValueError: Value not declarable with JSON Schema, field: name='_callbacks_List[langchain.callbacks.base.BaseCallbackHandler]' type=BaseCallbackHandler required=TrueI guess this could be fixed by setting infer_schema to False and explicitly passing the function schema excluding the callback field to the from_function method. I didn’t try this personally and instead used a slightly different approach.
Attempt 2 — Creating a tool from a function running the agent
Here, we simply define a function that runs the agent for us, and then build a tool from that function:
def prices_retrieval_tool():
"""
This function will help you find the price of a single inventory.
:return: the price
"""
def get_price(inventory_item: str) -> str:
json_agent = get_json_agent("./inventory_prices_dict.json")
result = json_agent_executor.run(
f"""get the price of {inventory_item} from the json file.
Find the closest match to the item you're looking for in that json, e.g.
if you're looking for "mahogany oak table" and that is not in the json, use "table".
Be mindful of the format of the json - there is no list that you can access via [0], so don't try to do that
""")
return result
price_tool = StructuredTool.from_function(func=get_price,
name='get inventory and furniture prices',
description='This function will help you get the price of an inventory or'
' furniture item.')
return price_toolwe can then go on and define an agent that uses this agent as a tool. As always, getting the prompt right for the agent to do what it’s supposed to do takes a bit of tweaking.
tools = [inventory_value_calculator_tool(), price_retrieval_tool()]
template = """
You should read a text and find all the inventory items that are mentioned to be in a room.
You have to print all the inventories that you find before getting the prices.
you have to get all the prices before computing the total price. Be mindful that some items may appear several
times and that you need to get a price for all items, so e.g. if five refrigerators are mentioned, you need the
value of the refrigerator five times.
Do not guess or estimate the price of the items or inventories. you must rely only on the prices that you get from the tools.
Use those prices to calculate the total value of items in the text using the tools provided to you.
Do not calculate the prices with anything else but the tools provided to you.
Begin!
Text: {input}
"""
llm = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo')
agent = initialize_agent(llm=llm,
tools=tools,
agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
verbose=True)
prompt = PromptTemplate(input_variables=['input'], template=template)
agent.run(prompt.format(input=STORY))When running this piece of code, the agent executes the price agent several times, successfully obtaining the price data from the JSON, and finally passing that input to our small sum calculator. Here is a small excerpt of the agent’s verbose output:
I need to continue finding the inventory items mentioned in the text and getting their prices. I can use regular expressions to find the items and the `get inventory and furniture prices` tool to get their prices.
Action:
```
{
"action": "get inventory and furniture prices",
"action_input": {"inventory_item": "rug of rich burgundy"}
}
```
> Entering new AgentExecutor chain...
Action: json_spec_list_keys
Action Input: data
Observation: ['sofa', 'television', 'refrigerator', 'microwave', 'bed', 'desk', 'chair', 'lamp', 'bookshelf', 'computer', 'coffee table', 'dining table', 'wardrobe', 'oven', 'dishwasher', 'washing machine', 'toaster', 'blender', 'rug', 'clock']
I see that there is a key for "rug", I should look at its keys to see if there is a match for "rich burgundy"
Action: json_spec_list_keys
Action Input: data["rug"]
Observation: ['price', 'volume']
Thought:I see that there is a key for "price" under "rug", I should use json_spec_get_value to get the value of "price"
Action: json_spec_get_value
Action Input: data["rug"]["price"]
Observation: 80.0
Thought:I have found the price of the rug of rich burgundy
Final Answer: 80.0
Thought:I need to continue finding the inventory items mentioned in the text and getting their prices. I can use regular expressions to find the items and the `get inventory and furniture prices` tool to get their prices.You can see how the main agent uses the agent tool, and how that triggers another AgentExecutor chain, which is exactly what we wanted.
If you go through the original text and sum up all the inventory items, you’ll end up at a total value of 1560. This is what our agent outputs at the end:
Observation: 1560.0
Thought:We have successfully calculated the total value of the inventory in the room. Now we can provide the final answer to the human.
Action:
```
{
"action": "Final Answer",
"action_input": "The total value of the inventory in the room is 1560.0."
}Summary
There are several ways to combine agents together and design them in a hierarchical way — I’ve provided one possible way to do it here, using a simple example. You can of course extend this approach to several layers of hierarchy to your own liking. Happy coding!
Credits to Hamid Khodabakhshandeh, with whom I explored this topic at a recent hackathon :)
