Mistral AI Function Calling — A Simple Example with Code
Introduction
Mistral is one of the major players in the AI world. Recently, they released the function calling API which, in other terms, is the API that enables the creation of assistants capable of actions (LAM). Function calling allows developers to make the logic and data of their applications accessible to LLMs.
The goal of this article is to explain, with examples, the interest of such an API, to detail the code that allows testing it, and finally to present a concrete demo video.
The code is available on GitHub, here.
In a previous article, I described how an API of this kind works, basing it on the one from OpenAI; the article can be found here.
How Mistral Function Calling works ?
The goal is as follows: a user interacts with a LLM in natural language in any language. Before providing the answer to the user, the LLM first checks in a library of functions (defined in advance by the developer) if one or more of them need to be executed to satisfy the user request. If so, then the LLM pauses and indicates to the system that it is waiting for the results of the pending functions. Once the results are sent, the LLM produces the final response to the user.
Thus, for this system to work, it is necessary for the developer to indicate to the LLM the set of available functions, their expected arguments, and of course, a natural language description of the functions and their arguments.
More concretely, the function calling API of Mistral AI is very simple (much simpler than that of OpenAI) and rather intuitive.
Here, it is not necessary to create an assistant or a thread; one executes a Mistral LLM with a list of messages and that’s it. Obviously, there are some specifics and rules to respect; all this is described in the following section that details the code.
Deep Dive into the Code
Use case
For the code presentation, let’s consider a simple use case: I’m developing a home automation assistant in natural language. I can manage different devices (door, light, etc.) from different zones (outdoor, kitchen, bedroom, etc.). The goal of the demo is to enable the user to perform several actions:
- List the available zones
- List the devices and their status in a zone
- Change the status of a device in a zone
In my example, the database is this JSON file:
{
1: {'zone': 'kitchen', 'devices': {'light': True, 'door': False}},
2: {'zone': 'outdoor', 'devices': {'light': True, 'camera': True}},
}Initialisation du service LAM
The beauty of this API is that it requires no special initialization, an empty conversation is sufficient.
def __init__(self):
current_app.logger.info(Config.MISTRALAI_KEY)
self.client = MistralClient(api_key=Config.MISTRALAI_KEY)
self.assistant_name = 'IT Administrator Assistant'
self.model_id= 'mistral-large-latest'
self.instruction= 'You are an IT administrator. You are responsible for managing user permissions. You have a list of users and their permissions. You can get the permissions of a user by their username, update the permissions of a user by their username, and get the user ID based on the username. You can use the following functions: getPermissionsByUsername, updatePermissionsByUsername, getUserIdByUsername.'
self.discussion = []Assistant functions
The developer must list the functions that will be accessible from the LLM.
To do this, the description of the functions is exactly the same as that defined by OpenAI. The developer must name the function, describe it in natural language, list and describe the expected parameters of the function.
def define_function__list_available_zones(self):
function = {
"type": "function",
"function": {
"name": "list_available_zones",
"description": "List the available zones of the house.",
"parameters": {
"type": "object"
}
}
}
return function
def define_function__list_device_status_by_zone(self):
function = {
"type": "function",
"function": {
"name": "list_device_status_by_zone",
"description": "List the status of devices in a specific zone.",
"parameters": {
"type": "object",
"properties": {
"zone": {"type": "string", "description": "The zone to list the device status for. Can be 'kitchen' or 'outdoor'."}
},
"required": ["zone"]
}
}
}
return function
def define_function__update_zone_device_status(self):
function = {
"type": "function",
"function": {
"name": "update_zone_status",
"description": "Update the status of a device in a specific zone.",
"parameters": {
"type": "object",
"properties": {
"zone": {"type": "string", "description": "The zone to update the status for. Can be 'kitchen' or 'outdoor'."},
"device": {"type": "string", "description": "The device to update the status for. Can be 'light', 'door', or 'camera'."},
},
"required": ["zone", "device"]
}
}
}
return functionLogic of the functions
Now, the developper needs to write the logic of the previously defined functions. Usually, developper should re-use existing services.
One specific information that is hidden in the following code: your functions should return textual content.
def list_available_zones(self):
current_app.logger.info(f'list_available_zones')
available_zones = current_app.domotics_service.list_available_zones()
if available_zones:
current_app.logger.info(f'Available zones found with ref: {available_zones}')
return json.dumps(available_zones)
else:
current_app.logger.info('No available zones found')
return "No available zone found"
def list_device_status_by_zone(self, zone):
current_app.logger.info(f'list_device_status_by_zone: {zone}')
devices = current_app.domotics_service.list_device_status_by_zone(zone)
if devices:
current_app.logger.info(f'Devices found')
return json.dumps(devices)
current_app.logger.info('No devices found')
return "No device found"
def update_zone_status(self, zone, device):
current_app.logger.info(f'update_zone_status: {zone}, {device}')
updated_status = current_app.domotics_service.update_zone_status(zone, device)
if updated_status:
current_app.logger.info(updated_status)
return json.dumps(updated_status)
else:
current_app.logger.info('Zone or device not found')
return "Zone or device not found"Run the assistant when a new user message is received
When a user sends a message, the following steps must be taken:
- Add the message to the list of messages (conversation)
- Execute the LLM by adding the available functions
- Add the LLM’s intermediate response to the message list
- Execute the pending functions (according to the LLM’s intermediate response)
- Add the functions’ responses to the list of messages
- Execute the LLM and return the response to the user
These steps are described in the following code:
def run_assistant(self, message):
current_app.logger.info(f'Running assistant: {message}')
self.discussion.append(ChatMessage(role="user", content=message))
ai_response = self.client.chat(
model=self.model_id,
messages=self.discussion,
tools=[
self.define_function__list_available_zones(),
self.define_function__list_device_status_by_zone(),
self.define_function__update_zone_device_status()
],
tool_choice="auto"
)
# Add the assistant response to the discussion
self.discussion.append(ai_response.choices[0].message)
# Check if there is a tool call in the response
tool_calls = ai_response.choices[0].message.tool_calls
if tool_calls:
current_app.logger.info(f'Tool calls found: {tool_calls}')
self.generate_tool_outputs(tool_calls)
current_app.logger.info(f'Discussion after tool call: {self.discussion}')
ai_response = self.client.chat(
model=self.model_id,
messages=self.discussion
)
current_app.logger.info(f'Assistant response after tool call: {ai_response.choices[0].message}')
self.discussion.append(ChatMessage(role="assistant", content=ai_response.choices[0].message.content))
current_app.logger.info(f'Assistant response: {ai_response.choices[0].message.content}')
return ai_response.choices[0].message.content
def generate_tool_outputs(self, tool_calls):
current_app.logger.info('Generating tool outputs')
tool_outputs = []
for tool_call in tool_calls:
function_name = tool_call.function.name
arguments = tool_call.function.arguments
args_dict = json.loads(arguments)
if hasattr(self, function_name):
function_to_call = getattr(self, function_name)
output = function_to_call(**args_dict)
current_app.logger.info(f'Tool output: {output}')
self.discussion.append(ChatMessage(role="tool", function_name=function_name, content=output))
return tool_outputsThe key thing to understand is that everything goes through adding messages to the conversation. Each message has a role, and as you can observe, the role can be “User”, “Assistant”, or “Tool”.
Video demonstration
A short demonstration in the following video:
Find the code in the following Github page.
Conclusion
In this article, I have presented a functional and concrete use case for Mistral AI’s function calling API. The API is excellent and intuitive. I wrote this article on March 27, 2024, and it is very likely that the API has evolved since then.
If you have any question about this post, contact me on LinkedIn.
