4 Practical GPT User Tips (Based on Actual Research)
Hey everyone, so I recently began a project to understand ChatGPT by reading the articles it recommended to explain itself. I’ve just passed the halfway point and it seemed like a good time to gather my thoughts. I’ve been seeing a lot of the following articles recently:
10 things you can do to optimize your ChatGPT Queries!!
Usually these articles seem like they have one of a few things in common.
a.) They are based off of gut feeling, “industry knowledge” or trial and error
b.) They are a list of specific queries you can use
c.) They are coming from folks with a non-data science background.
None of these things are bad or wrong, but I thought that I could give a different perspective based off of what I found while trying to explain the actual research papers behind the model. I’ve got 3 Do’s and 1 Don’t that I hope will give you a better understanding of how to best write a prompt in general. I’ll also be citing my sources so you can follow up if you don’t believe me. Let’s jump right in.
Make sure you subscribe so you can see all my stuff!
Give Examples
As I recently discussed in this article, GPT models and Large Language Models (LLMs) in general are based on the idea that with enough data and enough parameters, we can create a language model that can perform any task requested of it. There is no need for a huge training set of labeled data to teach it to do something new.
That being said if you look at the actual performance testing, the models do best at “few-shot learning.” All this means for practical purposes is that you give the model one or more examples of what you want. The model can respond without examples, but it does way better with them. Try a query like the one below for instance.
Can you help me think up some titles for a falafel recipe? For example, something like “Spice up your Life with this Spicy Falafel Recipe”
Source: “Language Models are Few-Shot Learners” by Brown et al. (2020)
Order Matters
One of the major improvements of transformers (the model structure that GPT is based on) over older architectures was that they don’t need to be fed each word in sequence. They can learn the dependencies between words using something called “positional encoding.”
In practice though, it has been found that LLMs struggle with long range dependencies. What this means is that the order you provide the information to the model can have an effect on its outcome (especially if the prompt is very long). If you give the question first, followed by a long block of text, it may forget the question. If you provide many examples, it may not weight the first example as much as the most recent one. It can also be useful to split things up. Provide the context for the question, and request a summary, then ask your question next.
Try multiple different orders like [Context, Question, Example] or [Question, Example, Context] to see which one gives you the best results. Or try splitting things up into Q1: [Context, Request for Summary] Q2:[Question, Example]
Sources: “Attention Is All You Need” by Vaswani et al. (2017)
“Language Models are Few-Shot Learners” by Brown et al. (2020)
Play to GPT’s Strengths
GPT and other LLMs were trained using a method called Generative Pre-Training. In a nutshell, they were fed chunks of incomplete text and asked to fill in the next word. What this means is that all LLMs actually do behind the scenes is generate the most likely next word based on the current context, one at a time.
While this behavior can be used to do all sorts of things, what LLMs are good at is continuing sentences. If you can’t or don’t want to provide an example then the next best thing is providing the beginning of a sentence and asking the model to finish it. For example:
Q1: Please summarize the following article [Article about Cooking]
Q2: Based on the above article, complete the following LinkedIn post: “Interested in trying something new? Check out…”
Source: “Improving Language Understanding by Generative Pre-Training” by Radford et al. (2018)
Don’t Ask for Comparisons
If you read the few-shot learners article and scroll down to the section on performance and limitations, one surprising thing you’ll find is just how bad LLMs are at comparisons. When asked if two words are used the same way in a sentence, or if one sentence implies another, it performs little better than chance.
What this means is that you really shouldn’t ask it to compare two things. Its better to ask specific questions about each and then compare the answers yourself to decide which you you prefer. For example instead of:
Which of these posts is better written [Post A] or [Post B]?
Try
Q1: Do you think [Post A] is well written? Can you follow the plot? What could I improve?
Q2: Do you think [Post B] is well written? Can you follow the plot? What could I improve?
Source: “Language Models are Few-Shot Learners” by Brown et al. (2020)
Conclusion
I hope you found at least one of these tips helpful! I also hope that it encourages you to try reading some of these scientific articles on your own. Or if you don’t want to you can subscribe to me and I’ll explain them for you!
The Author
With a Bachelors in Statistics, and a Masters of Data Science from the University of California Berkeley, Malachy is an expert on topics ranging from significance testing, to building custom Deep Learning models in PyTorch, to how you can actually use Machine Learning in your day to day life or business.
