ChatGPT — Prompt Engineering-Why is it Important ?

ChatGPT can be brilliant most times and then there are those other times when it says something dumb. This is where prompt engineering comes in. It’s not that your prompt wasn’t clear, but understanding the way ChatGPT was trained goes a long way towards crafting prompts that are going to yield useful outcomes.

Let’s start with understanding how the models are trained in the first place and the importance of Zero Shot, One-Shot and Few-Shot training.

Zero-Shot, One-Shot and Few-Shot training

Zero-shot, one-shot and few-shot learning are three types of learning methods in machine learning and computer vision. They describe how well a model can recognize and classify new objects or concepts with limited or no training examples.

Zero-shot learning refers to the ability of a model to recognize and classify new objects or concepts it has never seen before, relying on its understanding of related concepts. For example, a zero-shot learning model trained on different dog breeds might be able to classify a new breed it has never seen before based on its understanding of the attributes of dog breeds.

One-shot learning refers to the ability of a model to recognize and classify new objects or concepts with only one training example. For example, a one-shot learning model could be trained to identify an individual’s face based on a single image.

Few-shot learning refers to the ability of a model to recognize and classify new objects or concepts with a small number of training examples. For example, a few-shot learning model could be trained to identify different types of fruits with only a few training examples of each type.

The chart below from the paper: Language Models are Few-Shot Performers, shows the effectiveness of Few-Shot performance.

From the paper — Language Models are Few-Shot Learners July 2022

When ChatGPT says something dumb

Here is a very simple example, of when ChatGPT says something dumb. I got this example from a post on Medium and recreated it here. The task is clear and unambiguous, but providing the spelling of the word in the question is something you might assume that ChatGPT would not do. Actually it seems quite a dumb thing to do.

ChatGPT — Dumb response

You might also assume that asking ChatGPT to change this so that the spelling of the word is not in the question, would be helpful but it is not — it is actually very frustrating to employ this strategy. Try it for yourself.

Earlier we talked about zero-shot, one-shot and few-shot learning. What if we employed that strategy by providing an example of the outcome that we want in the prompt? Let’s start with a one-shot prompt.

ChatGPT — One Shot example

This is a better response, but what if I wanted the spelling test to include questions that include more than just animals? I can do this with a few shot example. See the response below.

Now it goes beyond the few examples I gave. It generates questions about words related to feelings, places not just things.

Prompt Engineering

This is a simple example of what we mean by the term Prompt Engineering. It’s more than being clear and unambiguous. It is an acknowledgement of sorts that we are not making this request of a human with general intelligence capabilities.

Prompt engineering in conversational AI involves the design, creation, and evaluation of prompts (questions or requests) for conversational AI models.

The goal of prompt engineering is to create high-quality, informative, and engaging prompts that can elicit relevant and accurate responses from conversational AI models.

Expectations

In all the hype around ChatGPT, it’s easy to forget that we are interacting with relatively narrow aspect of Artificial Intelligence.

ChatGPT is a type of AI model known as a Transformer-based language model, developed by OpenAI. It is trained on large amounts of text data to generate human-like text in response to prompts.

In the broader context of Artificial Intelligence (AI), ChatGPT falls under the category of Natural Language Processing (NLP), which involves teaching machines to understand, interpret, and generate human language. NLP is just one aspect of AI, which also includes fields such as computer vision, robotics, and expert systems.

In the context of Artificial General Intelligence, NLP models like ChatGPT are often referred to as narrow AI or weak AI, as they are trained for and excel at specific tasks but lack the general intelligence of a human.

If you are going to use ChatGPT effectively, or any AI model that is likely to emerge in the short term, you need to understand the emerging field of study known as prompt engineering.

None of this is to suggest that ChatGPT is not a catalyst for large scale change in the way we work. Hardly a day goes by when I am not using ChatGPT in one way or another, but it helps to know what it is, and what it isn’t.