Beyond Basics: Transforming Your Approach with Advanced OpenAI Function Calls and Tools
TL,DR; skip to advanced code examples if you don't want explanations.
Note: I have a lot of draft articles, when I picked up this article to publish, OpenAI recently added a new assistant API, and it's name from function calling to tools, I wrote mainly focused on the chat completion API in mind, but in true honesty, doesn't change the behaviour just the reserved word.
In June 2023, OpenAI introduced function calling (now called tools) into their API design; I rushed to build my first demo and propose for my team and a few startups, I'm mentoring. This advancement represents a significant leap from building basic demos to enabling large-scale, dynamic applications, since the return value from LLMs are never deterministic, only if you force to via cache or imposed constrain. It’s noteworthy that this "new feature" enables a bunch of new companies building plug-ins or GPTs on their platform, not to mention, I know a few that attracted VC capital to build wraps on top of this. In this article, I'll cover advanced ways of using this feature.
Brief intro on OpenAI API model function calling/ tools
OpenAI’s introduction of function calling marks a shift in AI interactions, differing from what we’re used to with services like Google or Siri. These services generally rely on simpler language processing techniques or are tailored to specific tasks, limiting their range of actions. Siri, for instance, is equipped for certain tasks and API interactions within the iOS system.
In contrast, OpenAI employs a more complex approach, using intent classification within its Transformer architecture. This method allows OpenAI’s models to understand and interpret a broader range of user requests. Building a dataset capable of training the models to recognise these diverse intents was challenging ( not cheap at all), just look at this open dataset that is probably covering 10% of what OpenAI model can. Despite this, OpenAI has succeeded in creating a system that can effectively grasp the context and intent behind user inputs.
This capability of OpenAI’s models to pinpoint and respond to particular functions based on word context enhances their utility. They can handle queries and tasks beyond their initial training, adapting to new situations and user inputs. This adaptability has opened up possibilities for more versatile and responsive applications, which many companies are beginning to explore and integrate into their services.
And last month someone, published a notebook attempting to implement this on Llama-2, an Open source Large language model by Meta.
What are the benefits?
It unlocks a few things and unblocks some current limitations like:
- Knowledge Cutoffs: It mitigates the issue of knowledge cutoffs by enabling real-time data access and processing, keeping the AI’s responses current and relevant.
- API Communication: The feature allows AI to communicate with both internal and external APIs, facilitating a seamless flow of information and functionality between different software systems.
- System Interactions: AI can now interact more effectively with various systems, enhancing its ability to perform tasks and automate processes.
- Personalised Responses: Function calling enables AI to provide customised responses by accessing and analysing user data, preferences, and history.
- Task Automation: It automates complex or repetitive tasks, making AI a valuable tool in streamlining workflows and operations across industries.
How it works?
Note: For practical reasons, I will focus on python 3.x code. This code works for OpenAI API and Azure OpenAI calls.
Simple Call
In this example, we provide additional payload to the body of the request as tools and we let the model decide which tool(function) to call, you can also prefixed function to always be called.
from openai import OpenAI
from dotenv import load_dotenv
import os
load_dotenv()
client = OpenAI()
client.key = os.getenv("OPENAI_API_KEY")
tools = [
{
"type": "function",
"function": {
"name": "get_weather_forecast",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
}
}
]
messages = [
{"role": "user", "content": "What's the weather like in Boston today?"}]
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=messages,
tools=tools,
tool_choice="auto"
)
print(completion)The basic principle is you signal (tools) to the model that you have a dynamic data and you want the model to enrich the response, inputing better representation of that data or interpretability of it in a matter that is more enriched. The other payload tool_choice to auto select which function to call and get the data from the system.
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=messages,
tools=tools,
tool_choice="auto"
)You can use like this for any of the following models:
- gpt-4
- gpt-4–1106-preview
- gpt-4–0613
- gpt-3.5-turbo
- gpt-3.5-turbo-1106
- gpt-3.5-turbo-0613
Scaling functions call
Now that you understand the functionality, you’ll see it’s straightforward and effective. However, scaling poses a challenge. Currently, we must manually map each function in a JSON dictionary or maintain consistent function naming for the model. This mapping is crucial for correctly identifying and executing the right function when it’s called. The main issue with this method is its limited scalability.
Python 3.5 introduced typed annotations to methods/functions back in 2006, let's try to take advantage of that and use the annotations to make the code more clean, maintainable and scalable, OpenAI fails to picture something like that in advanced function mapping examples in their cookbook, not sure why?
Creating an annotation python module, it's not hard, this will help us scale this, every time in my app I need to make that function discoverable by OpenAI API, and I just need to annotate and forget about it, since I always use the toot_choice to automatically select the function I need.
Functions metadata
from inspect import signature, Parameter
import functools
import re
from typing import Callable, Dict, List
def parse_docstring(func: Callable) -> Dict[str, str]:
"""
Parses the docstring of a function and returns a dict with parameter descriptions.
"""
doc = func.__doc__
if not doc:
return {}
param_re = re.compile(r':param\s+(\w+):\s*(.*)')
param_descriptions = {}
for line in doc.split('\n'):
match = param_re.match(line.strip())
if match:
param_name, param_desc = match.groups()
param_descriptions[param_name] = param_desc
return param_descriptions
def function_schema(name: str, description: str, required_params: List[str]):
def decorator_function(func: Callable) -> Callable:
if not all(param in signature(func).parameters for param in required_params):
raise ValueError(f"Missing required parameters in {func.__name__}")
@functools.wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
params = signature(func).parameters
param_descriptions = parse_docstring(func)
serialized_params = {
param_name: {
"type": "string",
"description": param_descriptions.get(param_name, "No description")
}
for param_name in required_params
}
wrapper.schema = {
"name": name,
"description": description,
"parameters": {
"type": "object",
"properties": serialized_params,
"required": required_params
}
}
return wrapper
return decorator_functionThis Python code functions_metada.py defines a decorator named function_schema, which is used to add a schema to functions. The schema includes the function's name, description, and details about its parameters. The decorator takes three arguments: name (the function's name), description (a brief description of what the function does), and required_params (a list of parameter names that are required for the function) and to describe each parameter we can take advantage of doc string if it's present in the function more below.
When you decorate a function with function_schema, it first checks if all required parameters are indeed part of the function's signature. If not, it raises an error. It then uses the parse_docstring function to extract descriptions of parameters from the function's docstring, assuming they are formatted as :param param_name: description. These descriptions are used to build a detailed schema of the function's parameters, which includes their type (assumed to be a string) and description.
Finally, this schema is attached to the function as an attribute named schema. This allows you to access detailed information about the function, like its parameters and their descriptions, programmatically. Such functionality can be useful for automatically generating documentation, validating function inputs, or building interfaces that interact with the function.
@function_schema(
name="get_weather_forecast",
description="Finds information the forecast of a specific location and provides a simple interpretation like, is going to rain, it's hot, it's super hot instead of warmer",
required_params=["location"]
)
def get_weather_forecast(location:Str)
return f"Forecast for {location} is ..."For example, now I just need to annotate any module function in my app, and OpenAI will be able to call it, now let's create the registry for those annotated functions, this will be a signalt o OpenAI on which function to call.
Now let's create a functions registry to automatically generate a map of all functions annotated with this method and generate the dictorinary used in the OpenAI chat completion API call.
Functions registry
Note: It’s great to map automatically the functions, I know that might be faster ways to map or load modules safely compared to this one. But is good enough.
import importlib.util
import os
from pathlib import Path
import json
import logging
from typing import Optional, Dict, List
logger = logging.getLogger(__name__)
class FunctionsRegistry:
def __init__(self) -> None:
self.functions_dir = Path(__file__).parent.parent / 'functions'
self.registry: Dict[str, callable] = {}
self.schema_registry: Dict[str, Dict] = {}
self.load_functions()
def load_functions(self) -> None:
if not self.functions_dir.exists():
logger.error(
f"Functions directory does not exist: {self.functions_dir}")
return
for file in self.functions_dir.glob('*.py'):
module_name = file.stem
if module_name.startswith('__'):
continue
spec = importlib.util.spec_from_file_location(module_name, file)
if spec and spec.loader:
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
for attr_name in dir(module):
attr = getattr(module, attr_name)
if callable(attr) and hasattr(attr, 'schema'):
self.registry[attr_name] = attr
self.schema_registry[attr_name] = attr.schema
def resolve_function(self, function_name: str, arguments_json: Optional[str] = None):
func = self.registry.get(function_name)
if not func:
raise ValueError(f"Function {function_name} is not registered.")
try:
if arguments_json is not None:
arguments_dict = json.loads(arguments_json) if isinstance(
arguments_json, str) else arguments_json
return func(**arguments_dict)
else:
return func()
except json.JSONDecodeError:
logger.error("Invalid JSON format.")
return None
except Exception as e:
logger.error(f"Error when calling function {function_name}: {e}")
return None
def mapped_functions(self) -> List[Dict]:
return [
{
"type": "function",
"function": func_schema
}
for func_schema in self.schema_registry.values()
]
def generate_schema_file(self) -> None:
schema_path = self.functions_dir / 'function_schemas.json'
with schema_path.open('w') as f:
json.dump(list(self.schema_registry.values()), f, indent=2)
def get_registry_contents(self) -> List[str]:
return list(self.registry.keys())
def get_schema_registry(self) -> List[Dict]:
return list(self.schema_registry.values())The FunctionsRegistry class is designed to streamline the management of dynamic function calls and their metadata in Python. It automatically scans a specified directory for Python modules and imports them, storing any callable functions that have a 'schema' attribute in a registry. This class is especially useful for handling functions dynamically, as it offers methods to resolve function calls with optional JSON-formatted arguments, generate a JSON file of all registered function schemas, and retrieve the contents of both the function registry and schema registry. Utilizing Python's standard importlib for dynamic module loading, pathlib for file system navigation, and logging for error reporting, this class represents a modern, robust approach to managing dynamic function calls.
We plan to utilize this class to pass all functions loaded from the ‘functions’ directory to the tools. It is compatible with asynchronous function types, which is particularly useful if you are working with streaming data. However, it’s important to note that for class methods, there might be limitations in using this approach to pass them directly as tools.
Advanced Functions/Tools calling
from typing import List, Dict, Any
from openai import OpenAI
from dotenv import load_dotenv
import os
import logging
from utils.functions_registry import FunctionsRegistry
# Set up logging
logging.basicConfig(level=logging.INFO)
def main() -> None:
load_dotenv()
try:
client = OpenAI()
client.key = os.getenv("OPENAI_API_KEY")
if not client.key:
raise ValueError("API key not found in environment variables.")
tools = FunctionsRegistry()
messages: List[Dict[str, str]] = [
{"role": "user", "content": "what's the weather forecast for Wellington, New Zealand"}
]
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=messages,
tools=tools.mapped_functions(),
tool_choice="auto"
)
print(completion)
logging.info(completion)
except Exception as e:
logging.error(f"An error occurred: {e}")
if __name__ == "__main__":
main()The simple call for functions registry is basically to mention to OpenAI that we have those functions available and the model will make the decision on which to call, it doesn't mean it will call it right way, we need to resolve the function or eval the function to be able to get the results and enrich back to the model, for now is able to identify which function to call from the map we sent to them, the output is something like this.
function_calling % python3 advanced_function_calling.py
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
INFO:root:ChatCompletion(id='chatcmpl-8SEoAV4YKjm7Ng2dKHOIvyTs4gb6k', choices=[Choice(finish_reason='tool_calls', index=0, message=ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_IeWbpcF3zw7DBWF3JXi8rRoN', function=Function(arguments='{\n "location": "Wellington, New Zealand"\n}', name='get_weather_forecast'), type='function')]))], created=1701740798, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=21, prompt_tokens=152, total_tokens=173))Now, let's enrich the response using the resolve function from Functions Registry, then we will be able to send back to OpenAI.
Scaling multiple parallel functions/tools calling
OpenAI provides a very simple implementation fixed on one idea of weather forecast, let's continue to use this example, but let's make more dynamic and able to execute any function** based on the name, imagine if we have to call multiple times the same, For instance, this example I've added a new method in my Functions Registry class.
def get_function_callable(self):
# Return a dictionary mapping function names to their callable functions
return {func_name: func for func_name, func in self.registry.items()}This would be useful in the example below.
from typing import List, Dict, Any
from openai import OpenAI
from dotenv import load_dotenv
import os
import logging
import json
from utils.functions_registry import FunctionsRegistry
# Set up logging
logging.basicConfig(level=logging.INFO)
def main() -> None:
load_dotenv()
try:
client = OpenAI()
client.key = os.getenv("OPENAI_API_KEY")
if not client.key:
raise ValueError("API key not found in environment variables.")
tools = FunctionsRegistry()
function_map = tools.get_function_callable()
# This might not work as the context is short, You have to stich up the prompt better to make it work in this particular example
messages: List[Dict[str, str]] = [
{"role": "user", "content": "Please provide the weather forecast for the following cities separately: Wellington, Auckland, and Christchurch in New Zealand."}
]
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=messages,
tools=tools.mapped_functions(),
tool_choice="auto"
)
response_message = completion.choices[0].message
tool_calls = response_message.tool_calls
if tool_calls:
messages.append(response_message)
for tool_call in tool_calls:
function_name = tool_call.function.name
if function_name in function_map:
function_args = json.loads(tool_call.function.arguments)
try:
function_response = function_map[function_name](
**function_args)
messages.append({
"tool_call_id": tool_call.id,
"role": "tool",
"name": function_name,
"content": function_response,
})
except Exception as e:
logging.error(f"Error in {function_name}: {e}")
second_completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=messages
)
logging.info(second_completion)
else:
logging.info(completion)
except Exception as e:
logging.error(f"An error occurred: {e}")
if __name__ == "__main__":
main()Conclusions
If you made till here, congratulations, We explore OpenAI’s function calling feature, which enhances AI application scalability. We discuss basic usage and address challenges in manual function mapping, you don't need a wrap for that. We introduce a Python function registry for automated schema generation and dynamic function resolution by name. This registry simplifies function cataloging and annotation, offering ease of access.
We demonstrate how the registry handles parallel calls to multiple weather functions, emphasizing the flexibility of OpenAI tools. It’s important to note that OpenAI tools, including the Assistants API, are currently in beta.
All the examples and code presented here, will be part of the OpenAI Cookbook and a new PR for the OpenAI Python package, this will help others be interested in this topic.
