A closer look to the new OpenAI GPT models ‘Function Calling’ Feature

In the dynamic landscape of Artificial Intelligence, where many giants play significant roles, one name sparking widespread conversation is OpenAI. Although it might not be the largest player, OpenAI has made substantial strides with its innovative Generative Pretrained Transformer (GPT) models.

These AI models have evolved (with GPT-4 being released this year in march and now accessible to users with succesful payment history) and with its latest versions (gpt-4–0613 and gpt-3.5-turbo-0613), OpenAI just introduced a new feature called ‘Function Calling’, taking AI’s capabilities to a new level.

Understanding Function Calling

This new feature basically allows developers to describe functions and have the model intelligently choose to output a JSON object containing arguments to call those functions. So in other words, it won’t magically call functions or apis directly but it allows you to create AI powered systems that can interact with others services based on the model responses.

On top of that, function calling also provides developers with the ability to extract structured data outputs from the model. This capability offers a systematic and orderly method for fetching pertinent information, thereby improving the usability and efficiency of the API across a diverse range of applications.

The synergy of smart function calling and the capacity to produce structured data responses paves the way for developers to tap into the power of OpenAI models and craft novel solutions.

Use Cases of Function Calling

The function calling feature enables developers to build chatbots that seamlessly integrate with external tools for query resolution.

For example, a request such as “Can you recommend a good Italian restaurant in Manhattan ? ” could be converted into a function call such as get_restaurants(country: string, zipcode: string, cuisine: string) allowing it to return an up-to-date list of restaurants based on some internal or external api call.

Likewise, requests such as “Could you book a table for two tonight 7pm at restaurant xyz?” could be processed using the following function call book_restaurant(restaurantId: string, when: string; guests: integer) where restaurantId would be known from previous api call.

Delegating function calling decision to GPT model

When being provided with functions description and schema, the GPT model will analyze the incoming messages and decide whether or not it would be appropriate to trigger one of the listed functions. If that’s the case, it will return the function name and a JSON object with parameters to be used to execute the given function. To generate a natural language response for the end user, an additional call to the GPT model is required. This call should include the results of the function call as input, allowing the model to convert the data into a coherent and conversational message.

Note: Functions are injected into the system message using a syntax familiar to the model. Consequently, these functions contribute to the model’s context limit and are considered input tokens for billing purposes. If you encounter context limits, we recommend either reducing the number of functions or shortening the documentation provided for function parameters.

How to achieve this in practice?

Suppose you have developed a frontend chatbot application and connected it to a Node.js serverless backend.

The initial step in this process involves creating a JSON schema for each function that you intend to expose to your chatbot. Using the aforementioned example, the schema might resemble the following:

{
 "name": "getRestaurants",
 "description": "Get restaurants matching request",
 "parameters": {
  "type": "object",
  "properties": {
   "country": {
    "type": "string",
    "description": "country in which the restaurant must be found. As a ISO 3166-1 alpha-2 code",
    "default": "US"
   },
   "zipCode": {
    "type": "string",
    "description": "zipcode in which the restaurant must be found. "
   },
   "cuisine": {
    "type": "string",
    "description": "type of found served in the restaurant ",
    "enum": ["ITALIAN", "CHINESE", "FRENCH", "THAI", "FUSION"]
   }
  }
 }
}

As for the bookRestaurants function, its JSON schema could be represented as follows:

{
  "name": "bookRestaurants",
  "description": "Book table at restaurant requested by the user",
  "parameters": {
    "type": "object",
    "properties": {
      "restaurantId": {
        "type": "string",
        "description": "restaurant id in which the reservation needs to be made",
        "example": " 273160a9-9195-4aae-abeb-6e21246761fc"
      },
      "when": {
        "type": "string",
        "description": "datetime of the the reservation. ISO 8601 format",
        "example": "2023-06-28T20:00:00.000Z"
      },
      "guests": {
        "type": "integer",
        "description": "Number of guests for the reservation",
        "example": 2
      }
    }
  }
}

Next, you are required to develop the handler code, which essentially serves as an orchestrator between the GPT API and the restaurant API. When making a call to the GPT API, you simply need to include your function schemas as an array in the functions parameter of the createChatCompletion method.

import {Configuration, OpenAIApi} from "openai";

import getRestaurantsSchema  from "../schema/getRestaurantsSchema.json" assert {"type":"json"};
import bookRestaurantsSchema  from "../schema/bookRestaurantsSchema.json" assert {"type":"json"};

import {getRestaurants, bookRestaurants} from "../service/restaurant";
const functions = {getRestaurants, bookRestaurants};


const configuration = new Configuration({apiKey: process.env.OPENAI_API_KEY});
const openai = new OpenAIApi(configuration);

exports.handler = async (event) => {
    try {
        const body = JSON.parse(event.body);

        const messages = body.messages;

        messages.push({"role": "user", "content": body.text})
        messages.push({"role": "system", "content": "today is " + new Date()})

        const response = await openai.createChatCompletion({
            model: "gpt-4-0613",
            messages: messages,
            functions: [getRestaurantsSchema, bookRestaurantsSchema]
        });

Subsequently, the model is responsible for determining whether or not a function should be invoked based on the user message that is passed. In our specific scenario, if we were to input the following message: “Can you recommend a good Italian restaurant in Manhattan?” The model would generate the following JSON object: { country: ‘US’, cuisine: ‘ITALIAN’, zipCode: ‘10001’ }

If the model determines that the request aligns with one of your functions, the response would include the name of the called function and the JSON object to be utilized for its invocation. Assuming you have created an object that lists all available functions, you can easily invoke the corresponding function or API using the arguments provided by the model in the form of a JSON object.

  let data = response.data.choices[0].message;
  
  if (data.function_call) {
      const functionName = data.function_call.name;
      const functionArgs = JSON.parse(data.function_call.arguments);
  
      const result = await functions[functionName](functionArgs);
  
      messages.push({
          "role": "function",
          "name": data.function_call.name,
          "content": JSON.stringify(result)
      })
  
      const second_response = await openai.createChatCompletion({
          model: "gpt-4-0613",
          messages: messages
      });
      data = second_response.data.choices[0].message;
  }
  
  return {
      statusCode: 200,
      body: JSON.stringify({"text": data.content, "messages": messages})
  }

} catch (e) {
    return {statusCode: 500, body: "Something went wrong"}
  }
}

Alternatively, you have the option to return both a completion and results, allowing you to exercise control over how they are displayed, such as in a list or grid format.

When it comes to the booking functionality, if your booking function requires ISO datetime information, you will need to remind the GPT model about the current date. This ensures that when you provide a request like “Please book a table for four tonight at 8 pm at…”, the model can accurately set the correct date. To achieve this, simply include a message with system role.

messages.push({role: "system", content: "today is " + new Date()});

Conclusion

The addition of these new features to the GPT models brings significant power, although occasional issues may arise as models can still encounter moments of hallucination. However, by implementing effective prompt engineering and meticulously defining schemas with descriptive properties and examples, you can achieve impressive outcomes. Naturally, there is a cost involved, but utilizing the “gpt-3.5-turbo-0613” version can help reduce it, even though it may lead to a potential decrease in the coherence and accuracy of the generated answers.

Personally, I am extremely delighted with this update, as I had previously attempted to accomplish similar results using older model versions and prompt engineering, but with significantly inferior outcomes. Now, achieving such results requires minimal effort and just a few lines of code. This development opens up a world of possibilities and opportunities.