Revolutionizing Language: GPT 3.5 Takes AI Communication to the Next Level!
GPT 3.5 Turbo was released by Open AI on 1st March 2023. ChatGPT, the conversational bot which put conversational AI into the headlines was powered using this model.
What is Turbo?
Open AI introduced the new family of models known as Turbo which are optimized for conversational chat input and output but also do equally well compared to the models previously released as part of GPT-3 with the most powerful one being DaVinci.
The below table shows an overview of how Turbo compares with previous models:
Open AI has released two models in the Turbo family, a stable release (gpt-3.5-turbo-0301) which has support for 3 months, and gpt-3.5-turbo which will get regular updates, similar to ChatGPT. Both models have a similar token limit as the previously released DaVinci, each new model has a maximum limit of 4096 tokens. As mentioned in my previous report, this means any output (completion) generated would be limited to 4096 minus the number of input tokens.
Open AI also claims the Turbo model family should be able to handle any use case already performed well by the DaVinci model family. The below screenshot shows how Turbo could be used for QnA in a similar way to how we used the DaVinci model in the previous report.
What has changed in terms of Data protection?
Contrary to the previous article which stated OpenAI will use data for model improvement, the new policy for OpenAI mentions that data will not be used to train or improve models unless opted-in by the customer. However, all data sent through the API will be retained for abuse and misuse monitoring purposes for a maximum of 30 days, after which it will be deleted (unless otherwise required by law). Currently the company stores all this data in the US which might be a potential GDPR issue with no data centre being available in Europe. As an exception, Open AI does also mention that customers deploying use cases with a low likelihood of misuse could request to not have API data stored at all, including for safety monitoring and prevention.
OpenAI also now offers dedicated instances for users who want more control over the model version and system performance. These instances allow developers to have full control over the load of the instance, enable features such as longer context limits, and pin the model snapshot. The dedicated instances are reserved for serving the developer’s requests and are paid for by time period. This gives developers deeper control and flexibility over their use of the OpenAI API however dedicated instances can make economic sense only when processing beyond ~450M tokens per day.
Cost Analysis
One significant advantage of the GPT 3.5 Turbo is that it is a tenth of the price of DaVinci. However, fine-tuning this model is not currently available. Pricing for Turbo is based on tokens (What is a token?) similar to older models and a user is charged every 1,000. Open AI estimates that around 750 words are equal to 1,000 tokens, so a paragraph consists of thirty-five tokens.
Open AI provides base models for implementation via its API.
Turbo is cheaper than DaVinci since it is optimized for conversation chat input. Since DaVinci requires more resources, the API call cost is higher. OpenAI suggests that for applications requiring a lot of understanding of the content, like summarization for a specific audience and creative content generation, DaVinci will produce the best results.
Turbo vs ChatGPT
In the previous report, we asked ChatGPT to write an article using a dataset (from Kaggle) containing the US unemployment rate. ChatGPT was able to identify data and statistical values and insights based on the input provided. However, the same was not possible on Turbo due to the token limitation of 4096 Tokens (see screenshot below).
From this, we can infer that the system design of ChatGPT allows it to exceed the token limit of the Turbo model (maybe through some form of data shrinking or selection). OpenAI suggests that if there are too many tokens to fit within a model’s maximum limit (e.g., more than 4096 tokens for gpt-3.5-turbo), the input has to be truncated, removed, or shrunk (Source).
Potential Use Cases
As suggested by Open AI, most use cases carried out by DaVinci models could be ported to Turbo. Here are some other use cases where Turbo could be useful:
- Customised chatbots: Turbo is a conversational model, the most common use case would be a customisable chatbot. Recently, Snap Inc. the creator of Snapchat released My AI for Snapchat+ users.
- Education/Training Platforms: Customisable chatbots could be created for education or training purposes to study and practice a particular subject. For example, Quizlet introduced Q-Chat which is an adaptive AI tutor that engages students with adaptive questions based on relevant study materials.
Conclusion and Further Steps
In conclusion, Turbo models provide on-par performance with DaVinci models, at 10% of the price. However, the token limit of 4096 tokens might be a limitation for some use cases, as seen in the example with the US unemployment rate dataset. Open AI has also updated its data protection policy, stating that data will not be used to train or improve models which is good news for enterprise users however their lack of data centres in Europe could lead to GDPR issues.
Overall, GPT 3.5 Turbo is a cost-effective option for conversational chat use cases, but DaVinci models might be better suited for tasks requiring a lot of understanding of content, like summarization or creative content generation.
Open AI also has made available its open-sourced Whisper API which is a speech-to-text model. We at Version 1 are currently analysing this feature and also looking at other emerging models of equivalent size or complexity and analysing how they perform against current state-of-the-art models. Stay tuned for more updates and visit the Innovation Labs to find out what Version 1 can do for you.
About the author:
Rohit Vincent is a Data Scientist here at Version 1.
Find out more about GPT-3 here.
