Unable to complete the tutorial for assistant APIs of Azure OpenAI Service
The original post (Japanese) was written on 23 February 2024.
Questions from a customer
One of my customers asked me about the following topic.
We’re currently evaluating the Assistant APIs of Azure OpenAI Service (AOAI). We just finished the tutorial using Python and AOAI directly. Now we’re evaluating the configuration where API Management (APIM) is in front of AOAI. At the moment, however, we are facing some problems that some actions can’t be done in this configuration. Could you please share the reasons and workarounds with us?
I asked them to explain the situation.
The tutorial they did is the following one.
There are two problems they’re facing.
- While using API key to access AOAI works fine, accessing with the managed identity of APIM does not work and returns 401 (
Permission denied). - Unable to get a sine wave image. No such REST API is found in the OpenAPI specification.
Reasons and workarounds
1. Error 401 (Permission denied)
Typically, we assign Cognitive Services OpenAI User role to APIM’s managed identity, but this is not appropriate for Assistant APIs. Because Assistant APIs create threads, assistants, etc., we need to assign Cognitive Services OpenAI Contributor role to the managed identity. Let me illustrate the API behavior with the screenshots of a REST client. Assume that you want to create an assistant using the following API.
1) Cognitive Services OpenAI User role is assigned to the identity of APIM.
As you can see, the API returns HTTP 401 (Permission denied), although the error message is not clear to diagnose the reason. This is because this operation of the API (create an assistant) needs contributor role.
2) Cognitive Services OpenAI Contributor role is assigned to the identity of APIM.
As you can see, the API returns HTTP 200.
We make sure that Cognitive Service OpenAI Contributor role is assigned to the managed identity of APIM. Additionally, Assistant APIs are stateful. If multiple AOAI instances are located behind a load balancer, configuration of session stickiness at load balancer is required.
2. No API to get file content is found in AOAI Assistant APIs.
In this context, we cannot find the following API in AOAI Assistant APIs, while the one is found in OpenAI APIs.
GET /files/{file_id}/contentIn fact, we cannot find it in the 2024-02-15-preview version.
On the other hand, this API is found in OpenAI and defined in the OpenAPI specifications.
The following is an except of OpenAPI specifications for OpenAI API.
...
/files/{file_id}/content:
get:
operationId: downloadFile
tags:
- Files
summary: Returns the contents of the specified file.
parameters:
- in: path
name: file_id
required: true
schema:
type: string
description: The ID of the file to use for this request.
responses:
"200":
description: OK
content:
application/json:
schema:
type: string
x-oaiMeta:
name: Retrieve file content
group: files
returns: The file content.
examples:
request:
curl: |
curl https://api.openai.com/v1/files/file-abc123/content \
-H "Authorization: Bearer $OPENAI_API_KEY" > file.jsonl
python: |
from openai import OpenAI
client = OpenAI()
content = client.files.retrieve_content("file-abc123")
node.js: |
import OpenAI from "openai";
const openai = new OpenAI();
async function main() {
const file = await openai.files.retrieveContent("file-abc123");
console.log(file);
}
main();
...Why on earth accessing AOAI directly through OpenAI Python library was successful? This is because direct access to AOAI works even if the API is not defined in OpenAPI specifications for AOAI APIs.
Currently, we need to add the API spec of this API with following OpenAI’s OpenAPI specifications to fix this. Here is an example after fixing API specifications. You can see we managed to get a “sine wave” image.
