Is it possible to check if Azure OpenAI Service REST APIs are healthy?

This entry is as of September 27, 2023, and the original article was published in Japanese.

Azure OpenAI Serviceの死活監視は可能なのか

Query

We’re configurating Azure OpenAI Service (AOAI) instances behind a L7 load balancer such as Application Gateway and Front Door to ensure availability and expand TPM. Is it possible to use a health probe equipped with L7 load balancer to check if AOAI APIs are healthy?

There seems no way to check if each AOAI instance is healthy from Application Gateway or Front Door because authentication is required to access AOAI.

Hint

When accessing REST APIs of AOAI, we see the response header of apim-request-id. This means that API Management (APIM) is one of components which configures AOAI. This APIM is not visible, and we cannot control it, but we might leverage this APIM to check REST API condition of AOAI.

How to

Invoking GET method to /status-0123456789abcdef… Quite simple. If you have configured Front Door in front of APIM, you would have read the document below. You don’t have to attach either bearer token or API key to call the operation.

GET https://<AOAI FQDN>/status-0123456789abcdef

Configure Azure Front Door in front of Azure API Management

Try it!

I used Postman to call the API operation. I could confirm the response 200.

Caution

We should keep in mind the following points.

• We can leverage this tip to check easily if AOAI APIs are accessible.
• Furthermore, please note that this way does not allow us to check if backend services of AOAI are healthy.