Claude 3: Function Calling in Action
Introduction:
OpenAI unleashed the power of function calling first, changing the game for how developers interact with LLMs. Function calling enables Large Language Models (LLMs) to connect with and effectively use external tools, facilitating interaction with external APIs. Similar to what we see in ChatGPT, where multiple tools can be used through chat (Data Analysis, Image Generation, etc.), we can give our agents access to tools we develop.
Anthropic’s Claude has emerged as a game-changer, offering developers and researchers a powerful suite of models to tackle complex tasks. In a recent announcement from Anthropic, Claude joined the function calling party. This allows developers to open up a world of possibilities for seamless data integration and task automation. In this article, we’ll dive into a codebase that showcases Claude’s function-calling prowess in the context of an NBA chatbot application.
Anthropic vs. OpenAI — Cost Comparison
For many categories, we’re seeing the Claude 3 family outperform GPT and Gemini. Not only are we seeing improvements across many benchmark tests in the industry, but we are seeing massive speed and price improvements. Haiku speeds are on par with GPT-3.5 and are consistently more accurate across the board.
Since the 2024 NBA playoffs are about to begin, I wanted to focus on the NBA for my application. To get some NBA data, we’ll be interacting with an API that makes it extremely easy to pull NBA statistics. When it comes to function calling, the primary thing to keep in mind is making your descriptions clear and concise:
tools = [
{
"name": "get_player_info",
"description": "Retrieves player information based on their name. Returns the player id and team",
"input_schema": {
"type": "object",
"properties": {
"DISPLAY_FIRST_LAST": {
"type": "string",
"description": "The name for the player."
}
},
"required": ["DISPLAY_FIRST_LAST"]
}
}
]In the example above, we are creating a list of tools that have a function named get_player_info . We describe this function that is clear and easy for the LLM to understand. Any tool that we create is tied to a real python function, so when the LLM identifies which tool to use, it can do so and have a repeatable output with ease:
def get_player_info(self, player_name):
players = commonallplayers.CommonAllPlayers()
players_df = players.get_data_frames()[0]
players_df = players_df[players_df['DISPLAY_FIRST_LAST'] == player_name]
return players_dfIn this function, we’re pulling in data based on the player's name. This is essentially the same as the tool we listed above. Now that there is a base understanding of how function calling works, let’s dive into the code:
import streamlit as st
import os
from dotenv import load_dotenv
import anthropic
from nba_api.stats.endpoints import commonallplayers
from nba_api.stats.endpoints import playercareerstats
from nba_api.stats.endpoints import franchisehistory
load_dotenv()
class AnthropicFunctionCalling:
def __init__(self):
self.API_KEY = os.getenv('ANTHROPIC_API_KEY')
self.MODEL_NAME = "claude-3-haiku-20240307"
self.client = anthropic.Client(api_key=self.API_KEY)
self.tools = [
{
"name": "get_player_info",
"description": "Retrieves player information based on their name. Returns the player id and team",
"input_schema": {
"type": "object",
"properties": {
"DISPLAY_FIRST_LAST": {
"type": "string",
"description": "The name for the player."
}
},
"required": ["DISPLAY_FIRST_LAST"]
}
},
{
"name": "get_player_statistics",
"description": "Retrieves the statistics of a specific player based on the player name. Returns the player ID, most points scored in a season, free throw pct for the same season, and the team the player is currently playing for.",
"input_schema": {
"type": "object",
"properties": {
"DISPLAY_FIRST_LAST": {
"type": "string",
"description": "The name for the player."
}
},
"required": ["DISPLAY_FIRST_LAST"]
}
},
{
"name": "get_league_titles",
"description": "Gets the number of league titles won by a team based on the team ID. Returns the team ID, team city, team name, start year, end year, and the number of league titles won.",
"input_schema": {
"type": "object",
"properties": {
"TEAM_ID": {
"type": "string",
"description": "The unique identifier for the team."
}
},
"required": ["TEAM_ID"]
}
}
]In the code above, we create the class AnthropicFunctionCalling, which allows us to set up all the variables that we’ll need for the rest of our app. We declare our API key, the primary model we’ll be using (Claude 3’s Haiku), the client, and our list of tools. Still, inside of this function, we’ll declare the 3 functions necessary to make the magic happen:
def get_player_info(self, player_name):
players = commonallplayers.CommonAllPlayers()
players_df = players.get_data_frames()[0]
players_df = players_df[players_df['DISPLAY_FIRST_LAST'] == player_name]
return players_df
def get_player_statistics(self, player_name):
player_info = self.get_player_info(player_name)
player_id = player_info['PERSON_ID'].values[0]
career = playercareerstats.PlayerCareerStats(player_id=player_id)
career_df = career.get_data_frames()[0]
career_df = career_df[career_df['PTS'] == career_df['PTS'].max()]
career_df = career_df[['PLAYER_ID', 'SEASON_ID', 'PTS', 'FT_PCT', 'TEAM_ID', 'TEAM_ABBREVIATION']]
return career_df
def get_league_titles(self, team_id):
history = franchisehistory.FranchiseHistory()
history_df = history.get_data_frames()[0]
history_df = history_df[history_df['TEAM_ID'] == int(team_id)]
history_df = history_df[['TEAM_ID','TEAM_CITY','TEAM_NAME','START_YEAR', 'END_YEAR', 'LEAGUE_TITLES']]
return history_dfAfter we’ve declared our functions, we need the ability to choose which tool to call. We’ll pass this to the LLM to ensure that it can process between the tools. Alongside this, we want to keep track of much money we are spending with each API call. We’ve created a function that calculates the cost of each API call:
def process_tool_call(self, tool_name, tool_input):
if tool_name == "get_player_info":
return self.get_player_info(tool_input["DISPLAY_FIRST_LAST"])
elif tool_name == "get_player_statistics":
return self.get_player_statistics(tool_input["DISPLAY_FIRST_LAST"])
elif tool_name == "get_league_titles":
return self.get_league_titles(tool_input["TEAM_ID"])
def calculate_cost(self, model_type, response_body):
if 'tools' not in response_body:
input_tokens = response_body.usage.input_tokens
output_tokens = response_body.usage.output_tokens
else:
input_tokens = response_body['usage']['input_tokens']
output_tokens = response_body['usage']['output_tokens']
# Define a dictionary with the cost per 1000 tokens for each model type
cost_per_thousand_tokens = {
"anthropic.claude-3-sonnet-20240229-v1:0": {"input": 0.003, "output": 0.015},
"claude-3-haiku-20240307": {"input": 0.00025, "output": 0.00125},
"claude-3-opus-20240229": {"input": 0.015, "output": 0.075},
"meta.llama2-70b-chat-v1": {"input": 0.00195, "output": 0.00256},
"ai21.j2-ultra-v1": {"input": 0.0188, "output": 0.0188}
}
# Get the cost per 1000 tokens for the given model type
cost = cost_per_thousand_tokens.get(model_type)
if cost is None:
raise ValueError(f"Invalid model type: {model_type}")
# Calculate the total cost
total_cost = cost["input"] * (input_tokens / 1000) + cost["output"] * (output_tokens / 1000)
return total_costPutting it all together:
Now that we’ve done the heavy lifting, we need some ability to interact with the app that we’ve created. We are going to be leveraging my favorite tool to demo data apps, Streamlit. Streamlit is a software product that allows you to share data apps in seconds and can bring LLM applications to life with ease:
def main():
st.title("NBA Chatbot")
avatar = "avatar.jpeg"
afc = AnthropicFunctionCalling()
if 'messages' not in st.session_state:
st.session_state.messages = []
if "dataframes" not in st.session_state:
st.session_state.dataframes = []
for message in st.session_state.messages:
with st.chat_message(message["role"], avatar=avatar):
st.markdown(message["content"])
prompt = st.chat_input("Write your question")
if prompt:
st.session_state.messages.append({"role": "user", "content": prompt})
with st.chat_message("user"):
st.markdown(prompt)
with st.chat_message("assistant", avatar=avatar):
response = afc.client.beta.tools.messages.create(
model=afc.MODEL_NAME,
max_tokens=4096,
tools=afc.tools,
messages=st.session_state.messages
)
initial_response_cost = afc.calculate_cost(response.model, response)
st.session_state.messages.append({"role": "assistant", "content": response.content})
while response.stop_reason == "tool_use":
tool_use = next(block for block in response.content if block.type == "tool_use")
tool_name = tool_use.name
tool_input = tool_use.input
tool_result = afc.process_tool_call(tool_name, tool_input)
st.session_state.messages.append({
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": tool_use.id,
"content": str(tool_result),
}
],
}
)
response = afc.client.beta.tools.messages.create(
model=afc.MODEL_NAME,
max_tokens=4096,
tools=afc.tools,
messages=st.session_state.messages
)
second_response_cost = afc.calculate_cost(response.model, response)
st.session_state.messages.append({"role": "assistant", "content": response.content})
final_response = next(
(block.text for block in response.content if hasattr(block, "text")),
None,
)
formatted_response = f"""
Based on the response, ensure to format the message as follows:
{final_response}
Rules:
- The message should be formatted as a string.
- The message should be a response to the user's question.
- The message should not mention the tool used.
- Do not include the xml tags in the response
- Respond in the exact format in the examples below
Example:
<get_player_info>"Draymond Greene currently plays for the Golden State Warriors."</get_player_info>
<get_player_statistics>"Here are some statistics regarding Draymond Greene:
1. The most amount of points he scored in a season was 100 and that occured during the 2020-2021 season.
2. His free throw percentage was 100% during the same season.
3. For the season he had his most points scored, he played for the Golden State Warriors.
"</get_player_statistics>
<get_league_titles>"The Golden State Warriors have won 6 league titles."</get_league_titles>
"""
st.session_state.messages.append({"role": "user", "content": final_response})
formatted_response = [{'role': 'user', 'content': formatted_response}]
final_output_response = afc.client.messages.create(
model='claude-3-opus-20240229',
max_tokens=4096,
temperature=0.0,
messages=formatted_response
)
final_output_response_text = final_output_response.content[0].text
st.markdown(final_output_response_text)
final_output_response_cost = afc.calculate_cost(response.model, final_output_response)
total_cost = initial_response_cost + second_response_cost + final_output_response_cost
st.session_state.messages.append({"role": "assistant", "content": final_output_response_text})
st.session_state.dataframes.append((prompt, total_cost))
for question, cost in st.session_state.dataframes:
with st.expander(f"Question - {question}"):
st.write(f"TOTAL COST: ${cost}")
if __name__ == "__main__":
main()Inside of our main function, we are instantiating our class and setting up the chatbot. There is a bit of prompt engineering in here, so I would definitely recommend checking out Anthropic’s prompt engineer guide which can be found here. After this occurs, we prompt the user what type of question they would like to ask and then begin sending requests to Claude:
In the screenshot above, we can see that just asking a question about statistics prompted the get_player_statistics function to be called. Claude was able to identify which tool to use and then return a response back to the user. To add to this, we can see the cost of the query in the screenshot as well.
Conclusion:
Automated function calling represents a pivotal advancement in the realm of application development, heralding a new era where efficiency and innovation converge. By seamlessly integrating with various tools and technologies, it not only promises to redefine the operational paradigms of numerous industries but also sets the stage for unprecedented growth and possibilities. As we stand on the cusp of this technological revolution, the anticipation for the transformative impact of automated function calling on our future workflows and industry standards is a testament to the promising horizon that lies ahead.
Get in Touch / Code Repo
Linkedin — https://www.linkedin.com/in/ryan-k-ba8274122/
Github Repository — https://github.com/klapp101/anthropic_function_calling/tree/main
