A review of the Azure Open AI Service
Azure provides an exclusive platform to use the OpenAI service directly from their Cognitive Service Platform. Users can access the service through REST APIs, Python SDK, or a web-based interface (screenshot below) in the Azure OpenAI Studio. For other on-premise or cloud platforms, one would have to deploy a script using their SDK which will call the OpenAI API services.
Below are the features available on the service:
For the models available, we note that GPT 3.5 Turbo is not available on the Azure Open AI service at the moment. All other Open AI models are available to use with the addition of all other features the cognitive service provides such as Virtual private networks, Managed Identity services, and Private Endpoints.
The regional availability of the Azure service is limited for now and should grow in the future. We can see from the above table that fine-tuning for Davinci and Cushman models is available only in the South-Central US region. These region-based limitations might create GDPR issues if a business wants to store its data locally. However since fine-tuning might not be needed for most general business scenarios, Azure does provide the West Europe region for enterprises based out of Europe.
Azure OpenAI Service Studio
Azure has released the OpenAI Service studio which allows you to do the following:
- Experiment with prompt completions using their own version of Playground.
- Customize a model with fine-tuning.
- Create multiple deployments to compare models or deploy them for different use cases.
- Manage the performance results of custom models fine-tuned on datasets.
Azure's own version of Playground as shown above allows you to test out different models deployed for various examples. The features include what is available in the OpenAI playground. The only significant difference in the Azure playground is the lack of accessibility to the latest models such as Turbo which should be released shortly.
Pricing Analysis
Given those models which are fine-tuned would be hosted on Azure servers, the price of fine-tuned models has an additional hosting charge per hour compared to the Open AI pricing (mentioned in our previous blog).
For base models, both the Azure service and OpenAI have similar price structures for inference. However, for fine-tuned models (given Azure already charges you for hosting the model) the Open AI service is more expensive — as you can see from the chart below. A fine-tuned DaVinci model on the Azure service would provide more than 80% savings compared to what Open AI provides.
Which is better? Open AI or Azure?
The prices shown in the table below are purely an assumption based on pricing structures (OpenAI, Azure) and could vary. The table shows the prices for fine-tuning a model in terms of training and inference for different token lengths.
Note: Processing time for the above chart is assumed to be 1 hour for all token sizes.
Open AI seems to be the better choice if the model you are training uses a dataset with approximately 100k tokens. However, for larger requirements where we process around 3 million tokens, Azure would be cheaper overall.
What do we do next?
In conclusion, the Azure OpenAI Service provides access to OpenAI models, including GPT-3, through a REST API, Python SDK, or web-based interface. However, the latest models such as the GPT-3.5 Turbo are not available yet on the platform. It also offers additional features which support enterprise users such as virtual network support, managed identity, and private link support. The Azure OpenAI Service Studio allows users to experiment with prompt completions, customize models with fine-tuning, and manage performance results.
In terms of pricing, Azure and OpenAI have similar pricing structures for base models with Azure providing additional security features such as private networks and endpoints. However for fine-tuned models, Azure charges an additional hourly fee for hosting, but the cost of training and inference for these models is generally lower than OpenAI’s pricing structure. Overall, using Azure Open AI Service can provide infrastructure security and cost savings for fine-tuning models on very large datasets, compared to using OpenAI’s services directly. The limited regional availability of the fine-tuning service may be a consideration for some users, particularly those subject to GDPR compliance requirements. However, the base model for general scenarios is available in West Europe giving Enterprise users control over where the data is sent.
Find out more about the Version 1 Innovation Labs here.
About the author:
Rohit Vincent is a Data Scientist at Version 1.
