Overview of Open AI LLM Function Calling
This is the first article in a series about LLM Agents. Before diving into this complex topic, it’s crucial to understand the essential component of agents: Function Calling.
Function Calling enables LLMs to perform tasks by invoking specific functions, making it a cornerstone for building intelligent LLM Agents capable of handling multiple tasks seamlessly.
Each article comes with a notebook that lets you try and experiment with the concepts on your own.
Open AI Function Calling
The latest versions of gpt-35-turbo and gpt-4 are fine-tuned to work with functions and are able to both determine when and how a function should be called.
The model chooses a function to invoke and also identifies the function parameters from the text context.
Model returns a JSON object that contains the function name and the arguments for the selected function.
Note that you have to make these calls yourself, as the models only generate them.
This model skill is essential for developing LLM Agents, which could use multiple function calls to produce the final outcome.
Let’s begin.
Setup and Configuration
To run the code, you need to have an Azure subscription and create an Azure Open AI service. Then, you need to configure your environment variables with the Azure Open AI API Key and the Azure Open AI Endpoint.
#install openai SDK
%pip install -q openai python-dotenv# Configure Azure Open AI
os.environ["AZURE_OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
os.environ["AZURE_OPENAI_ENDPOINT"] = os.getenv("OPENAI_DEPLOYMENT_ENDPOINT")
# Details about supported API versions: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference
os.environ["OPENAI_API_VERSION"] = "2024-02-01"
os.environ["OPENAI_API_TYPE"] = "azure"Be aware that this post is based on the most recent (as of writing this post) Azure Open AI API version: “2024–02–01” which supports the latest Open AI models.
I also employ two Open AI LLM models: gpt-4-turbo-2024–09–04, which is the most recent one at the time of writing this post, and gpt-4-turbo-1106.
Function Calling
from openai import AzureOpenAI
llm = AzureOpenAI()Below is a code that defines a function called “NER” with the model parameter “tools”.
The function identifies Named Entities in the input text.
The Named Entities are names of persons, organizations, and job roles.
The function description is important here, because it tells the model what to do, like a system prompt.
You can also add description to each function parameter to help the model understand more information.
The JSON format for the “tools” parameter is commonly used by OpenAI for Function Calling.
response = llm.chat.completions.create(
model="gpt-4-2024-09-04",
temperature=0,
messages=[
{"role": "system", "content": "Think carefully and then analyze the text as instructed. If a text doesn't contain any named entities, just provide an empty object."},
{"role": "user", "content": "My name is Vlad and I'm working at Microsoft as a Solutions Architect."}
],
tools=[
{
"type": "function",
"function": {
'name': 'NER',
'description': 'Extract all named entities in the provided context. NER stands for Named Entity Recognition.',
'parameters': {
'type': 'object',
'properties': {
'person': {'description': 'person name', 'type': 'string'},
'company': {'description': 'company name', 'type': 'string'},
'job_title': {'description': 'person job title', 'type': 'string'}
},
'required': ['person', 'company', 'job_title']
}
}
}
]
)You can create more than one function, as we will demonstrate later in this article.
Keep in mind that the “tools” tokens counts toward the model maximum context size.
Let’s look at the response details.
response.choices[0].finish_reason'tool_calls'When the model chooses to call a function, the “finish_reason” is “tool_calls”.
And this is a function name to be called:
response.choices[0].message.tool_calls[0].function.name'NER'Moreover, the model will recognize the function arguments in the input text, doing Named Entity Recognition for this particular example.
response.choices[0].message.tool_calls[0].function.arguments'{"person":"Vlad","company":"Microsoft","job_title":"Solutions Architect"}'A helpful feature is that we can use Function Calling to ask models for different tasks, such as entity recognition, key phrases extraction, sentiment identification, translation, and more.
The main benefit of using Function Calling is that both the input (the “tools” parameter) and output are structured data, which makes it simpler and more reliable than working with free text.
How about input text with no Named Entities? What will the model do?
response = llm.chat.completions.create(
model="gpt-4-2024-09-04",
temperature=0,
messages=[
{"role": "system", "content": "Think carefully and then analyze the text as instructed. If a text doesn't contain any named entities, just provide an empty object."},
{"role": "user", "content": "Hello, how are you today?"}
],
tools=[
{
"type": "function",
"function": {
'name': 'NER',
'description': 'Extract all named entities in the provided context. NER stands for Named Entity Recognition.',
'parameters': {
'type': 'object',
'properties': {
'person': {'description': 'person name', 'type': 'string'},
'company': {'description': 'company name', 'type': 'string'},
'job_title': {'description': 'person job title', 'type': 'string'}
},
'required': ['person', 'company', 'job_title']
}
}
}
]
)# Checking the finish reason and the function call
if response.choices[0].finish_reason == 'stop':
print("The model chose to stop invoking functions and output the final answer.")In this situation, the value of “finish_reason” is “stop”.
response.choices[0].finish_reason'stop'This means the model chooses to stop invoking the functions and output the final answer.
response.choices[0].message.content"I'm here to help you! How can I assist you today?"In this case the content has the model output. If a model chooses to use a function, the content will be empty.
Multiple Function Calling
If you want to do sentiment analysis besides named entity recognition, you can make another function for that.
I added a new function called SentimentAnalysis:
response = llm.chat.completions.create(
model="gpt-4-turbo",
temperature=0,
messages=[
{"role": "system", "content": "Think carefully and then analyze the text as instructed."},
{"role": "user", "content": "My name is Vlad and I'm working at Microsoft as a Solutions Architect. I love OpenAI."}
],
tools=[
{
"type": "function",
"function": {
'name': 'NER',
'description': 'Extract all named entities in the provided context. NER stands for Named Entity Recognition.',
'parameters': {
'type': 'object',
'properties': {
'person': {'description': 'person name', 'type': 'string'},
'company': {'description': 'company name', 'type': 'string'},
'job_title': {'description': 'person job title', 'type': 'string'}
},
'required': ['person', 'company', 'job_title']
}
}
},
{
"type": "function",
"function": {
"name": "SentimentAnalysis",
"description": "Extract sentiment in the provided context. The sentiment value must be one of these: `positive`, `negative`, or `neutral`.",
"parameters": {
"type": "object",
"properties": {
"sentiment": {"description": "sentiment value", "type": "string"}
},
"required": ["sentiment"]
}
}
}
]
)Note that as of this article’s writing, gpt-4-turbo-2024–09–04 didn’t support multiple function calls, so I use another model for this example: gpt-4-turbo version 1106.
response.choices[0].finish_reason'tool_calls'The model decides to invoke two functions: ‘NER’ and ‘SentimentAnalysis’ and it sets up the appropriate arguments for each function, performing Entity Recognition and Sentiment Analysis.
for func in response.choices[0].message.tool_calls:
print(func.function.name)
print(func.function.arguments)NER {"person": "Vlad", "company": "Microsoft", "job_title": "Solutions Architect"}
SentimentAnalysis {"sentiment": "positive"}The output is a perfectly valid JSON, making it easily convertible to a Python dictionary, which is far more efficient and less error prone than dealing with unstructured output.
str_out = response.choices[0].message.tool_calls[1].function.arguments
d = json.loads(str_out)
d["sentiment"]'positive'Conclusion
Function Calling is a key feature for creating LLM agents, enabling structured input and output for reliable performance in complex NLP tasks. In this blog, we’ve explored how OpenAI implements Function Calling to perform Named Entity Recognition and Sentiment Analysis. Future articles will delve into integrating this feature with frameworks like LangChain, further enhancing the capabilities of LLM agents.
Stay tuned for more advanced techniques and applications.
What’s coming next …
We will improve the existing implementation by using LangChain and other excellent and popular LLM frameworks. Keep following.
Here is the complete code version: https://github.com/vladfeigin/llm-factory-pub
