OpenAI — Prompt Engineering for GPT3
The Large Language Models (LLM) is the most searched topic by data scientists and data science enthusiasts in the current era. LLM are the state-of-art models used for various Natural Language Processing (NLP) and Machine Learning (ML) tasks. GPT-3 from OpenAI has created a paradigm shift in the world of NLP. GPT-3 is the first step toward making technology accessible to everyone. It enables people from all walks of life to solve complex technical problems using a user-friendly interface. With GPT-3, training prompts can be designed for specific AI problems using natural language, without needing to know the technical details.
GPT-3
GPT-3 is a game-changer in the world of Natural Language Processing. With its ability to return a text completion in the natural language given any text input (prompt) like a phrase or sentence, GPT-3 is revolutionizing the way we interact with computers. OpenAI offers four typesGPT-3 models with different trained parameters and specifications. For more details on GPT-3 models, it’s working and customizing GPT-3 for a specific application, check out this interesting blog!
Prompt Engineering
A prompt is a key input to GPT-3 for getting a favourable and contextual response. Thus, prompt design is the most significant process in GPT-based applications.
The prompt design is like playing the game of charades!
The good prompts need research and understanding about how GPT-3 is trained, what it knows about the world and how it can be utilized to produce contextual responses. While playing the game of charades, the person is given enough information to figure out the word using intelligence. Similarly, GPT-3 is given just enough context in the form of a training prompt to figure out the patterns and perform the given task (completion, Q&A, prediction, etc. as per requirements). GPT-3 has been pre-trained on a vast amount of text from the open internet. There are different kinds of inputs:
- Zero-shot learning: Direct Input to the model
- One-shot Learning: Input with one training example
- Few-shot Learning: Input with a few training examples
As a rule of thumb: zero-shot → one-shot → few-shot → corpus training
Following is the step-by-step approach to designing an optimal prompt:
Step-1: Define your application and categorize it into a natural language processing task such as classification, chatbot, completion, etc.
Step-2: Try zero-shot learning to get an optimal response from GPT-3 as per requirements without feeding any training examples to the model
Step-3: If there’s a need for training examples, iterate over Step 2 to rethink strategies and solutions with zero-shot learning
Step-4: If few-shot learning is the only solution, use the OpenAI playground to design your prompt with different experiments according to the application context
Step-5: Feed as few as possible examples to avoid overfitting the model or skewing the predictions
Prompt Examples
The prompts for GPT-3 depend on the type of application. This section shows some examples to have a better idea about how to design the prompt for the specific application.
Classification
Prompt:
Decide whether a Tweet's sentiment is positive, neutral, or negative. Tweet: "I loved the new Batman movie!"
Sentiment:
Q&A
Prompt:
I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer. If you ask me a question that is nonsense, trickery, or has no clear answer, I will respond with "Unknown".
Q: What is human life expectancy in the United States?
A: Human life expectancy in the United States is 78 years. Q: Who was president of the United States in 1955?
A: Dwight D. Eisenhower was president of the United States in 1955.
Q: Which party did he belong to?
A: He belonged to the Republican Party. Q: What is the square root of banana?
A: Unknown Q: How does a telescope work?
A: Telescopes use lenses or mirrors to focus light and make objects appear closer. Q: Where were the 1992 Olympics held?
A: The 1992 Olympics were held in Barcelona, Spain. Q: How many squibs are in a bonk?
A: Unknown Q: Where is the Valley of Kings?
A:
Thus, the prompts depend on the type of application but the previous section shows the best practices that can be applied to any use case. The OpenAI provides a playground to experiment with different prompts and optimize them for your application.
Conclusion
NLP has revolutionized the world and GPT-3 will take it further to the next step. The development of GPT-3 applications, will redefine the way we look at technologies and interact with machines. There is so much more to do with GPT-3 but for sure it will work for a better future in natural language processing and generation.
Want to customize GPT-3 for your application? Check out here.
Stay tuned!
Follow me: M. Haseeb Hassan
