A Journey into Prompt Engineering Techniques
Welcome to our blog, where we’ll dive into the fascinating world of prompt engineering and explore various types of prompts that can supercharge your interactions with language models. As artificial intelligence continues to advance, harnessing the power of language models has become increasingly important. However, simply inputting a query may not always yield the desired results. That’s where prompt engineering comes in — the art of crafting effective instructions or cues that guide language models to produce the desired output. In this article, we’ll uncover the secrets behind successful prompt engineering and delve into different types of prompts, ranging from basic techniques to more advanced strategies. Get ready to unlock the full potential of language models and take your AI interactions to the next level. Let’s embark on this journey of discovery and empowerment in the realm of prompt engineering!
Prompt engineering (PE) refers to the effective communication with AI systems in order to achieve desired outcomes. With the rapid advancement of AI technology, the skill of mastering prompt engineering has become increasingly valuable. The application of prompt engineering techniques spans across a wide range of tasks, making it a valuable tool for enhancing efficiency in both every day and innovative activities.
With prompt engineering we can automate tasks that you spend countless hours doing right now. It could be writing emails, writing reports, or even writing code. If you can describe the task to an AI, it can likely either do it for you or at least give you a starting point.
What is prompting?
The process of instructing an AI to do a task is called prompting. We tell the AI a set of instructions (the prompt) and it performs the task. Prompts can be as simple as a instruction/question, or as complex as huge chunks of text.
Here are two examples of prompts:
1) Summarizing an Article
Say you are reading an article about snowfall in Florida. You want to quickly get the main ideas of the article, so you show the AI what you are reading, and ask for a summary:
2) Math Word Problem Solving
If you have a dataset of mathematical equations that you would like a language model to solve, you can create a prompt by posing the question “What is EQUATION”.
For a given question, your full prompt could look like this:
For this prompt, ChatGPT (an AI) sometimes answers 569,050 (incorrect). This is where prompt engineering comes in.
CAUTION: One more thing, if you are running the prompts in GPT-3, you should set temperature to 0 to remove randomness.
There are many types in prompting, below are some of them:
Instruction Prompting
One of the simplest prompting methods is just giving instructions (sometimes called instruction prompting). modern AIs can follow much more complex instructions.
Example 1
ChatGPT can understand much more complex instructions, the example below shows that:
It is very exciting to see how the model can extrapolate from the instructions. For example, it knows to replace Cheap Dealz with [COMPANY] and Jimmy Smith with [SENDER], even though we did not explicitly tell it to do so.
Caution: Allowing AI to remove PII from text is a promising approach, but it should be used with extraordinary caution as it may make mistakes.
Role Prompting
Another prompting technique is to assign a role to the AI. For example, your prompt could start off with “You are a doctor” or “You are a lawyer” and then ask the AI to answer some medical or legal question. Here is an example:
Example 1
This is a correct answer, but if the AI had simply been prompted with What is 100*100/400*56?, it would have answered 280 (incorrect). Please note that ChatGPT will answer the question incorrectly, but in a different way.
When assigning a role to the AI, we are giving it some context. This context helps the AI understand the question better. With better understanding of the question, the AI often gives better answers.
Note: This technique is no longer as effective with more modern AIs (e.g. GPT-3 davinci-003). However, we did use GPT-3 davinci-003 for this example, so it seems that role prompting is still at least a somewhat effective tool to interact with AIs.
Example 2
Example 3
Few shot prompting
Yet another prompting strategy is few shot prompting, which is basically just showing the model a few examples (called shots) of what you want it to do.
Example 1
Consider the above example, in which we are attempting to classify customer feedback as positive or negative. We show the model 3 examples of positive/negative feedback, then we show it a new piece of feedback that has not been classified yet (It doesnt work!:). The model sees that the first 3 examples were classified as either positive or negative, and uses this information to classify the new example as negative.
The way that we structure the examples is very important. Since we have structured these 3 examples as input: classification, the model outputs a single word after the final line, instead of saying a full sentence like this review is positive.
More on structure
A key use case for few shot prompting is when you need the output to be structured in a specific way that is difficult to describe to the model. To understand this, let’s consider a relevant example: say you need to compile names and occupations of well known citizens in towns nearby by analyzing local newspaper articles. You would like the model to read each article and output a list of names and occupations in First Last [OCCUPATION] format. In order to get the model to do this, you can show it a few examples:
Example 2
Variants of shot prompting
The word “shot” is synonymous with “example”. Aside from few-shot prompting, there are two other types of shot prompting that exist. Do not overthink this! The only difference between these variants is how many examples you show the model.
Variants:
- 0 shot prompting: no examples are shown to the model
- 1 shot prompting: 1 example is shown to the model
- few shot prompting: 2+ examples are shown to the model
0-shot prompting
0-shot prompting is the most basic form of prompting. It is simply showing the model a prompt without examples and asking it to generate a response. As such, all of instruction and role prompts that you have seen so far are 0-shot prompts. An additional example of a 0-shot prompt is:
As you can see, there is some level of explanation from chatGPT like “The multiplication of 3 multiplied by 3 is” with this prompt. It is 0-shot since we have not shown the model any complete examples.
1-shot prompting
1-shot prompting is when you show the model a single example. For example, the 1-shot analogue of the 0-shot prompt Add 2+2:
We have shown the model only 1 complete example (Add 3+3: 6), so this is a 1-shot prompt.
Few-shot prompting
Few-shot prompting is when you show the model 2 or more examples. All prompts above this variants section have been few-shot prompts. The few-shot analogue of the above two prompts is:
As you can see the results from the above two prompts are more precise with no explanation and this approach can be used for complex prompts as well. Few-shot prompting is a powerful technique for getting the model to produce accurate and properly formatted output!
Combining Techniques
As we have seen in the previous pages, prompts can have varying formats and complexity. They can include context, instructions, and multiple input-output examples. However, thus far, we have only examined separate classes of prompts. Combining these different prompting techniques can lead to more powerful prompts.
Here is an example of a prompt that includes context, instructions, and multiple examples:
By adding additional context/examples, we can often improve the performance of AIs on different tasks.
We have now covered multiple types of prompts, as well as ways to combine them. This page will provide you with terms to explain different types of prompts. Although there have been approaches to formalize discourse around prompt engineering1, the field is ever changing, so we will provide you with just enough information to get started.
Parts of a Prompt
There are a few different parts of a prompt that you will see over and over again. They are roughly:
- A role
- An instruction/task
- A question
- Context
- Examples (few shot)
We have covered roles, instructions, and examples in previous pages. A question is simply a question! (E.g. what is the capital of France?). Context is any relevant information that you want the model to use when answering the question/performing the instruction.
Not all of these occur in every prompt, and when some do occur, there is no standard order for them. For example, the following two prompts, which each contain a role, an instruction, and context, will do roughly the same thing:
You are a doctor. Read this medical history and predict risks for the patient:
January 1, 2000: Fractured right arm playing basketball. Treated with a cast.
February 15, 2010: Diagnosed with hypertension. Prescribed lisinopril.
September 10, 2015: Developed pneumonia. Treated with antibiotics and recovered fully.
March 1, 2022: Sustained a concussion in a car accident. Admitted to the hospital and monitored for 24 hours.January 1, 2000: Fractured right arm playing basketball. Treated with a cast.
February 15, 2010: Diagnosed with hypertension. Prescribed lisinopril.
September 10, 2015: Developed pneumonia. Treated with antibiotics and recovered fully.
March 1, 2022: Sustained a concussion in a car accident. Admitted to the hospital and monitored for 24 hours.
You are a doctor. Read this medical history and predict risks for the patient:However, the second prompt is likely preferable since the instruction is the last part of the prompt. This is good since the LLM is less likely to simply write more context instead of following the instruction. For example, if given the first prompt, the LLM might add a new line: March 15, 2022: Follow-up appointment scheduled with neurologist to assess concussion recovery progress.
A “Standard” Prompt
We have heard of a few different formats of prompts thus far. Now, we will quickly jump back to the beginning and define a “standard” prompt. Following Kojima et al.2, we will refer to prompts that consist solely of a question as “standard” prompts. We also consider prompts that consist solely of a question that are in the QA format to be “standard” prompts.
Why should I care?
Many articles/papers that we reference use this term. We are defining it so we can discuss new types of prompts in contrast to standard prompts.
Two examples of standard prompts:
Standard Prompt
What is the capital of France?Standard Prompt in QA format
Q: What is the capital of France?
A:Few Shot Standard Prompts
Few shot standard prompts3 are just standard prompts that have exemplars in them. Exemplars are examples of the task that the prompt is trying to solve, which are included in the prompt itself4. In research, few shot standard prompts are sometimes referred to simply as standard prompts (though we attempt not to do so in this guide).
Two examples of few shot standard prompts:
Few Shot Standard Prompt
What is the capital of Spain?
Madrid
What is the capital of Italy?
Rome
What is the capital of France?Few Shot Standard Prompt in QA format
Q: What is the capital of Spain?
A: Madrid
Q: What is the capital of Italy?
A: Rome
Q: What is the capital of France?
A:Few shot prompts facilitate “few shot” AKA “in context” learning, which is the ability to learn without parameter updates
LLM Settings
The output of LLMs can be affected by configuration hyperparameters, which control various aspects of the model, such as how ‘random’ it is. These hyperparameters can be adjusted to produce more creative, diverse, and interesting output. In this section, we will discuss two important configuration hyperparameters and how they affect the output of LLMs.
Temperature
Temperature is a configuration hyperparameter that controls the randomness of language model output. A high temperature produces more unpredictable and creative results, while a low temperature produces more common and conservative output. For example, if you adjust the temperature to 0.5, the model will usually generate text that is more predictable and less creative than if you set the temperature to 1.0.
Top p
Top p, also known as nucleus sampling, is another configuration hyperparameter that controls the randomness of language model output. It sets a threshold probability and selects the top tokens whose cumulative probability exceeds the threshold. The model then randomly samples from this set of tokens to generate output. This method can produce more diverse and interesting output than traditional methods that randomly sample the entire vocabulary. For example, if you set top p to 0.9, the model will only consider the most likely words that make up 90% of the probability mass.
Token length
This is the number of words or characters in a sequence or text that is fed to the LLM.It varies depending on the language and the tokenization method used for the particular LLM.The length of the input text affects the output of the LLM.A very short input may not have enough context to generate a meaningful completion. Conversely, a rather long input may make the model inefficiently process or it may cause the model to generate an irrelevant output.
Max tokens
This is the maximum number of tokens that the LLM generates.Within this, is the token limit; the maximum number of tokens that can be used in the prompt and the completion of the model. Determined by the architecture of the model LLM, it refers to the maximum tokens that can be processed at once.The computational cost and the memory requirements are directly proportional to the max tokens. Set a longer max token, and you will have greater context and coherent output text. Set a shorter max token, and you will use less memory and have a faster response but your output is prone to errors and inconsistencies.During the training and fine-tuning of the LLM, the max token is set.Contrary to fine-tuning token length during the generation of output, the coherence and length of the output is carefully set at inception, based on the specific task & requirements, without affecting other parameters that will likely need adjusting.
Stop tokens
In simple terms, it is the length of the output or response of an LLM. So it signifies the end of a sequence in terms of either a paragraph or a sentence. Similar to max tokens, the inference budget is reduced when the stop tokens are set low. For example, when the stop tokens are set at 2, the generated text or output will be limited to a paragraph. If the stop tokens is set at 1, the generated text will be limited to a sentence.
Temperature, top p, and other model configuration hyperparameters are key factors to consider when working with language models. By understanding the relationship between these hyperparameters and the model output, practitioners can optimize their prompts for specific tasks and applications. Some models, like ChatGPT, don’t let you adjust these configuration hyperparameters (unless you use the API).
Conclusion
Having said all of this, it’s important to note that what has been discussed so far is merely scratching the surface. The aforementioned techniques are just rudimentary prompts, while there exist numerous intricate prompting techniques that will be covered in the upcoming blog. It’s worth noting that these Language Models (LLMs) are already surpassing human performance, as evident from the provided image. If you’re interested in delving deeper into prompting and learning about state-of-the-art techniques and their applications, I highly recommend visiting an incredible website that has been a major source of information for this blog.
References
- Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020). AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10.18653/v1/2020.emnlp-main.346 ↩
- This paragraph is from https://en.wikipedia.org/wiki/Snow_in_Florida↩
- Prompting | Learn Prompting: Your Guide to Communicating with AI
- Top P, Temperature and Other Parameters | by Dixn Jakindah | May, 2023 | Medium
