🤯AI: Foundation Models
Foundation Models are Mind Blowing
[This article is a part of the 🤯AI series]
Foundation models like GPT-3 and Codex are models trained on broad data at scale so they can be adapted to a wide range of downstream tasks. We will cover how the Azure OpenAI Service enables you to use language models for text and code generation, reasoning, inferencing and comprehension and how you can customize these powerful models with few shot learning using your labeled data.
The Stanford Institute for Human-Centered Artificial Intelligence’s (HAI) Center for Research on Foundation Models (CRFM) coined the term foundation model. This term refers to “any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks”. At their core, these models are based on deep neural networks and self-supervised learning.
Foundation models are “designed to be adapted (e.g., finetuned) to various downstream cognitive tasks.
We will focus on Foundational Language Models. The recent mind-blowing capability of these foundation models is due in no small part to the industry’s ability to build and train models with a massive number of parameters, just look at this hockey stick growth in the number of parameters available in recent years:
What’s in the box with Azure OpenAI Language Models?
GPT-3 base series: understand and generate natural language
Codex series: understand and generate code from natural language
Embeddings series: understand and use embeddings
One way to think about the model options in Azure Open AI is to view them on a spectrum of model “power”, where speed and cost increase from A to D:
- Ada (least powerful, fastest, least expensive)
- Babbage
- Curie
- Cushman
- Davinci (most powerful, slowest, most expensive)
The power of completions
OpenAI’s models can do everything from generating original stories to performing complex text analysis, “completing” text based on a “prompt”.
The completions endpoint provides a text-in, text-out interface
- Prompt: Your input text
- Completion: Generate text that attempts to address your prompt
Applications:
- Classification
- Generation
- Conversation
- Transformation
- Summarization
The following screenshots shows an example using the Open AI Studio Playground with the Tldr; prompt that requests a summary of paragraph.
The responses the foundation model returns can be surprising and mind-blowing. For example, if I try a little Monty Python humor:
Prompt: What is the unladen airspeed of a European swallow?
Response: The unladen airspeed of a European swallow is about 11 meters per second, or 24 miles per hour.
Wow.🤯
Want to try this for yourself? Follow the simple steps in this document to access the Open AI Studio environment:
