Perfecting powerful prompts: what we learnt from our interactive workshop on generative AI
This article follows on from the interactive skills-sharing workshop that took place on Wednesday 19th April, 2023, at Artlist, in Norwich.
Introduction
To recap my introduction to the topic of generative AI and prompt engineering:
You don’t need technical skills to use generative AI
It’s generally even easier than using Google! However, some technical explanation is helpful:
Conversational, text-based, generative AI (like ChatGPT) relies on a technology called a ‘large language model’ (LLM). LLMs are a type of machine learning ‘neural network’ that is trained on text data found across the World Wide Web on the Internet. We’re talking billions and trillions of words from online articles, forum posts, Wikipedia entries, etc.
In theory LLMs just predict the next likely word in a sentence, but the emergent property of intelligence means that the more text information the LLM gobbles up, the more it learns about the world in general. This is what I think is so ground-breaking about the recent developments in AI: I might describe these new LLMs as devices for converting linguistic, semantic meaning into machine-operable logic. Imagine if you could have a chat with an entity born from the sum knowledge of the World Wide Web itself…
Introducing our digital friends
So far, only a handful of LLM chatbots are publicly available; I’ve experimented with ChatGPT, Bard, and Bing Chat.
ChatGPT is the most well-known, and often the most useful. However, currently ChatGPT can’t search the web to find out new things, which is a drawback to be aware of, seeing as the data it was trained on only extends to September 2021 — it won’t know about anything that happened after that cut-off point (unless you tell it). However, ‘plugins’ are planned for ChatGPT that will let it search the web, among other features. ChatGPT is free but you can pay a monthly subscription of $20 to make use of a more powerful version. Go to chat.openai.com to start using ChatGPT.
Bard is Google’s answer to ChatGPT. Bard can search the web to improve its answers, but it just isn’t as smart as ChatGPT right now (sorry, Bard!). Bard will generally provide you with three answers to your prompts; from what I can tell, the first answer is based on Bard’s training data, the second answer is based on an internet search, and the third answer is a synthesis of the first two, but confirmation of this online is hard to come by. Google Bard is free to use. Go to bard.google.com to start using Bard.
Bing Chat is Microsoft’s AI-powered search functionality. You can access it in the Bing smartphone app or in Microsoft’s Edge browser. Bing Chat is powered by OpenAI’s ‘GPT’ model of LLM (which also powers ChatGPT). Bing Chat is free to use, but will sometimes throw adverts into its answers. You can learn more about how to start using Bing Chat here.
If you’re not sure which AI chatbot to try out, I recommend starting with the free version of ChatGPT.
What is a ‘prompt’?
A prompt is a bit like the query you enter into a search engine. At it’s core, a prompt is just some text input you enter into an LLM in order to ‘prompt’ it to respond.
- A bad prompt will get you rubbish output.
- An ok prompt will get you ok output.
- A good prompt can (potentially) get you phenomenal output.
We want to write good prompts.
Writing good prompts, a general guide
- Talk to AI like a colleague, not a ‘machine’. LLMs have been trained on discursive content like online forum conversations, so if you want a well-written answer, it helps to provide a well-written question.
- Be very specific. The more information you give an LLM, the more likely it will give you the exact output you are after. Remember, the World Wide Web is a big place! Try to narrow down the subject area you want to find out about.
- Give it examples (if you want). LLMs are generally fast learners and aim to please. Providing a couple of examples of the kind of output you are after will go a long way.
Creating a ‘continuous semantic context’
Imagine you have two friends: Bob and Sally.
Bob and Sally have never met each other.
You send Bob a message:
“Hi Bob, it’s my birthday next Thursday! I’m going to the pub.”
Then you message Sally:
“Would you like to come along?”
Sally has no idea what you are talking about. You haven’t told Sally about the pub, or that it is your birthday.
This is how conversations work with ChatGPT. It’s a bit like having separate message threads with each of your friends in your instant messaging app of choice (WhatsApp, Telegram, etc).
To illustrate this, I started a new chat with ChatGPT, and gave it some context for a sentence, then asked it what it thought this sentence meant:
Clearly ChatGPT picks up what we’re on about. Notice that the idea of Dave being a dog and the idea that someone called ‘Dave’ is poorly with arthritis occur in two separate sentences. This isn’t an issue for ChatGPT; it follows along perfectly.
Next, I started a brand new chat and only provided the same sentence, without explaining the context of poor Dave the fictional dog and his medical woes.
We can clearly see that ChatGPT has no idea what I’m on about, other than the plausible guess that Dave is some bloke.
This is an important concept to get your head around. If I keep talking to ChatGPT in the same chat thread, ChatGPT can read all of our messages so far in that chat thread to give me better answers.
My recommendation is that you use the same chat thread for the same topic because ChatGPT can be more helpful the more it understands. For example, if I were working on a project, I would tell ChatGPT what kind of project I’m doing, what it’s for, what my deliverables are, etc. This will help ChatGPT tailor it’s output to be more helpful.
On the other hand, it’s easy to ‘pollute’ your semantic context, by introducing new concepts into an existing chat thread. This might confuse ChatGPT, and lead to less accurate results. For example, if I had a pet snake that is also called Dave, ChatGPT might get confused as to whether I’m talking about a snake or a dog. If I had two completely different projects (or two separate but quite similar projects), I might choose to talk about each project in a separate chat thread, in order to maintain a consistent semantic context.
Beware ‘hallucinations’!
Currently, the biggest drawback of LLMs is that they will ‘fill in the blanks’ if they don’t know something — often in very plausible sounding ways! LLMs can struggle to know what they know and what they don’t know (as an aspect of meta-cognition). The best remedy for this is improved access to information. Think of an LLM like a very enthusiastic intern on the first day of work: they really want to impress you, but sometimes they make mistakes!
Testing ‘use cases’
Article summarisation
The first thing we tried was summarising a long article. We found this article on Time Magazine’s website.
When you make a new chat thread with ChatGPT, you can choose which ‘model’ to use.
GPT4 is the most powerful, but it is the slowest to respond (although answers are still given within a few minutes at most). GPT3.5 is faster, but not as smart.
ChatGPT-3.5 made the rooky mistake of thinking it knew something about an article written after its training data cut-off date of September 2021, and seems to have summarised a completely different article (that may or may not exist).
Bard unfortunately didn’t fare much better, and although the format of the output responds well to specific requests (like a ‘bullet point breakdown’), it describes books that are not mentioned in the article!
Bing Chat didn’t fair much better, and has similarly made up a synopsis about various books that do exist, but were not discussed on that page.
However, going to the article in question, and asking Bing to summarise ‘the current page’ does give us an accurate result:
Being able to ask suggested follow-up questions is a nice feature.
Heading back to ChatGPT, we found that GPT-4 (the most powerful model, available through paid subscriptions) twigged that it wasn’t trained on that article, as it must have been published in or after 2022!
What we did to get round this was to copy and paste the entire contents of the webpage (CTRL + A, CTRL + C, CTRL + V) that contains the article and dump it into the text input field of ChatGPT’s user interface.
The result is an accurate summary; bang on.
Taxonomical classification
Can AI classify items according to an existing classification?
We jumped straight in and established if ChatGPT-3.5 knows about classifying species of organism:
This approach introduces the overall use case to ChatGPT gradually, so we can be more confident the results will be factually accurate.
We asked whether it is capable of fulfilling the task we want, before actually asking it to accomplish that task:
Attempting to emulate a plausible real-life situation for this use case, I made up an encounter with an as-yet unidentified creature:
The output from this prompt hasn’t provided the full taxonomy of our loud feathered friend, but we can just ask:
Googling and checking a few results suggests this classification is completely accurate. You can even ask for ChatGPT’s results as a table or a chart:
Making webpages on command
This is a great party trick (depending on the kinds of parties you get invited to). The language-oriented nature of LLMs means they often excel at writing programs in a range of computer languages. LLMs are great at reading through long, complicated documentation, and can easily explain what a bunch of code is doing, and suggest improvements.
You can copy the output from the chat thread, open up a code editor, and paste the output into a new HTML file.
You can add interactive functionality to these pages, in a <script> tag, and more visual styling in a <style> tag. Obviously you don’t have to generate the whole webpage in one go; if you want to spend time perfecting one part, you can focus ChatGPT on that and paste code it has generated into your existing document.
Honestly, this is one of the most exciting use cases for generative AI for me — although also one of the most daunting; anyone with an internet connection now has access to a free online tool that is approximately as good at coding as a junior developer. If you pay your $20 per month, you can access GPT-4, and you will be astonished at how competent it is. Remember: the key to good results is good prompting!
Conclusion
ChatGPT-4 is by far the most accurate conversational AI chatbot currently available. Issues around summarising online articles can be mostly circumvented by pasting raw text into the input field of ChatGPT’s webpage, although this feels awkward.
Bing Chat will have some use cases, and could be a useful tool to help you engage with the content of webpages further, but is limited compared to the wider general functionality of ChatGPT. Google’s Bard isn’t powerful enough yet to successfully compete with GPT LLM models when tested and compared.
If you go for a walk and find an unusual beastie, ChatGPT should be able to find you the exact species classification from a plain text description — this opens the possibility of other kinds of classification, of objects, concepts, etc.
Making simple webpages with usable interactivity takes a few seconds, which opens up more possibilities for ‘no code’ editors and lowers the barriers to creating expressive online content. There’s never been a better time to learn to create computer programs yourself, and I predict an explosion in bespoke software tools and products from people who might previously have had only a passing interest in computer science.
We also tested Midjourney, the AI image generator, with the use case of generating cover images for blog posts. I’ve previously covered Midjourney here. Our conclusion is that you absolutely can make fantastic images to go along with articles and blog posts, and this is very easy to accomplish. I recommend adding --v 5 to your prompts, so that you can use version 5 of Midjourney’s image creation engine, which is significantly more powerful than version 4, and can create convincingly photo-realistic images with ease.
Smaller details can be unconvincing (the policeman furthest right has a squashed head) and hands are still somewhat tricky, but otherwise this is plausibly a photograph of real people in a real place:
Something to notice about Midjourney's output is that although I used the keywords "police officers", all the AI-generated figures are white policemen; there has been discussion about biases and stereotypes being perpetuated through generative AI, and the nature of an averaging data set tends towards showing 'default' results. To what extent existing biases will be recreated by automated processes is an increasingly important topic to consider. Greater awareness of prompt engineering may be one approach to address this--having used Midjourney a lot before, I'm reasonably confident that we could specify extra keywords relating to gender and ethnicity, and the image generator would happily oblige us with the content we want. Ultimately neural nets will recreate the ethical context of the data they are trained on, so how we filter and manage data in relation to existing power structures is an important topic.
Further resources
You can learn more about the newly expanding field of ‘prompt engineering’ by checking out online tutorials and guides; two website I’d recommend looking at are:
Codecademy has just released a free, 7-hour course on using ChatGPT, which includes information on prompt engineering: codecademy.com/learn/intro-to-chatgpt
The Norfolk Network will be holding a panel discussion on the business implications of AI on June 8th 2023, at Artlist (Norwich).
☕ If you found this article interesting or helpful, you can buy me a coffee to support my caffeination needs: ko-fi.com/hortfrancis
