How to force GPT models to always use functions
Integrating OpenAI models into your products can open up a realm of possibilities, but it also poses unique challenges. Often, you’ll find yourself in situations where the integration process requires external tools. I.e. creating an agent or chatbot.
One classic instance of this is the knowledge retrieval agent, designed specifically for Q&A applications that operate over extensive sources, like large PDFs. OpenAI saw the need and rolled out function calling. This feature lets you describe functions, so “the model can intelligently output a JSON object with the right arguments for those functions”. It’s a smooth link between GPT’s capabilities and external tools and APIs.
If this sounds like Greek to you, consider diving into these resources before reading on:
- Function Calling and Other API Updates by OpenAI
- OpenAI API Reference on Function Calling
- How to Call Functions for Knowledge Retrieval: A Guide
Forcing Function Calls: The How-To Guide
The world of chatbots and agents is diverse. In some scenarios, every user input should translate to an API call. How do you ensure this?
You can specify the functions calling behaviour using function_call parameter. According to the docs, function_call:
Controls how the model responds to function calls. none means the model does not call a function, and responds to the end-user. auto means the model can pick between an end-user or calling a function. Specifying a particular function via {“name”: “my_function”} forces the model to call that function. none is the default when no functions are present. auto is the default if functions are present.
But here’s the catch: Using the function_call in ‘auto’ mode doesn’t guarantee a function call every time. There might be instances where the model opts for a text response.
While prompt engineering can offer a partial solution, it doesn’t entirely eliminate the risk of the model bypassing a function call.
The ‘Divide and Conquer’ Strategy with ReAct Prompting
Ah, the age-old ‘divide and conquer’ approach! A cornerstone in the software world, it once again proves invaluable here.
To ensure your chatbot or agent consistently calls a function, consider this two-step process:
- Function Selection: This initial step determines which function to invoke based on the input. By leveraging an enum parameter in the function call, you can guarantee the model selects one of the provided functions. If this sounds complex, rest assured — the coding part is a breeze!
- Function Invocation: Following the selection, this step defines the parameters required to execute the chosen function.
Not only does this approach elevate the reliability of your agent or chatbot, but it also offers a straightforward way to implement the ReAct framework if we include a thought parameter in the first completion request:
import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")
system_message = """
You are a very helpful assistant. Your job is to choose the best posible action to solve the user question or task.
These are the available actions:
- calculator: run a calculation on two numbers
- get_weather: get the current weather
- google_search: get the top 5 results on google
"""
user_input = "What is the weather like in Barcelona?"
agent_function = {
'name': 'select_action',
'description': 'Selects an action',
'parameters': {
'type': 'object',
'properties': {
'thought': {
'type': 'string',
'description': 'The reasoning behind the selection of an action'
},
'action': {
'type': 'string',
'enum': ["calculator", "get_weather", "google_search"],
'description': 'Action name to accomplish a task'
}
},
'required': ['thought', 'action']
}
}
completion = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
temperature=0,
messages=[
{"role": "system", "content": system_message},
{"role": "user", "content": user_input}
],
functions=[agent_function],
function_call={'name': 'select_action'}
)
print(completion['choices'][0]['message']['function_call']['arguments'])
# Output:
#
# {
# "thought": "The user wants to know the current weather in Barcelona.",
# "action": "get_weather"
# }Now we have the name of the function for the function_call parameter for the following step, we can ask the model to give us the necessary arguments for the function.
def get_weather(city: str) -> str:
return f"The weather in {city} is awesome ☀️"
get_weather_description = {
"name": "get_weather",
"description": "Retrieve the weather for a specific city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "Name of the city to get the weather for"
}
},
"required": ["city"]
}
}
completion = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
temperature=0,
messages=[
{"role": "user", "content": user_input}
],
functions=[get_weather_description],
function_call={'name': 'get_weather'}
)
print(completion['choices'][0]['message']['function_call']['arguments'])
# Output:
#
# { "city": "Barcelona" }import json
arguments = completion['choices'][0]['message']['function_call']['arguments']
get_weather(**json.loads(arguments))
# Output:
#
# 'The weather in Barcelona is awesome ☀️'We are currently using this approach with great results in a production environment and we have tested this framework with more than 7 tools successfully.
The main drawback of this approach is the increase in token usage but the there is no clean alternative at the moment with a single call. Also, separating the tasks into two steps increases the model success rate as suggested by OpenAI Best Practices (https://platform.openai.com/docs/guides/gpt-best-practices/strategy-split-complex-tasks-into-simpler-subtasks)
Here at Magnettü 🧲, we’re leveraging the power of the most recent OpenAI models, incorporating them using the approaches we’ve outlined above. We’re always open to learning and adapting, so please share your experiences and methods of integration in the comments 🙆🏼♂️.
