OpenAI’s New Algorithm Can Write Your Essay Better Than You — and Much More

Shoot! Your eyes dart to the clock, “10:39 PM”. It completely slipped your mind. “I have to get this essay done” you think to yourself. Another hour passes, and midnight only comes closer as you stare at your blank document. Until this very moment, you’ve been the student of dreams, the goody-two-shoes of the group. However, as you approach the last ten minutes before your deadline, the desperation becomes unbearable. Swiftly switching tabs, you search: “Essay writer”. Clicking the first resource you could find, you enter the following into the site: “Please write a short, roughly 500 word essay about Shakespeare. Keep the language simple and concise”, (you tried to be as specific as you could, really put your personal *flare* into your computer written masterpiece). A few seconds later, you’re met with a simple, concise essay about Shakespeare. No errors, not too wordy or pretentious, she’s beautiful. You don’t have time to sit and awe, though, and slap your name on the title before submitting. But wait, what just happened?

The start of an AI revolution: GPT-3

Little did you know, that essay was generated on the spot. It wasn’t made by any other human, and is unique to what you asked. This site used a new, highly advanced AI algorithm to write it, known as Generative Pre-Trained Transformer 3 (this isn’t straight out of Transformers, I promise), or GPT-3. Put into simple terms, GPT-3 is an AI language model that uses its existing knowledge (more on that later) to predict what word, or in this case sequence of words, is most likely to come after another.

It’s similar to the autocomplete feature on our smartphones, in that they both use probability to determine what phrases or words will follow another. However, there’s one key difference between these two technologies. Notice that if you repeatedly use the autocomplete feature on iMessage (Android users are welcome to leave), it begins to build completely incoherent sentences, ones that sound nothing like you. GPT-3 has the ability to emulate human language almost perfectly using a single prompt or key word, just like with your essay. Achieving this was no walk in the park, though.

A sentence written using the iMessage autocomplete feature.

How it does this (TW: Many terms and definitions)

In order for GPT-3 to truly emulate human language, we need to trace language back to its origins, the brain. Since computers don’t possess brains like ours, ones built upon billions of neurons, GPT-3 mimics one through a neural network. Neural networks impersonate the web of neurons in our brain through mathematics, in GPT-3’s case, through probability. To build this web, GPT-3 undergoes a process called machine learning of which there are two forms: supervised and unsupervised learning.

Supervised learning, courtesy of the name, is much more tedious than the latter. It’s a process where humans manually label thousands of data sets and meticulously describe each respective output for an input, which the computer then learns from. For example, IMAGNET spent years and hundreds of employees labelling each and every characteristic of an image in order for a computer to do the same, known as computer vision. Fei Fei Li, a researcher who worked with the company, highlighted a great example of this. In order for their algorithm to recognize a single element of an image, for example a cat, people worldwide worked to label millions of cats as cats, factoring in every colour, lighting, or position a cat could be in. So while supervised learning provides much liberty and control to ones making these advances, it’s expensive, and extremely time taking.

Many consider GPT-3 to be so advanced because it uses unsupervised learning (I’ll stop trashing on supervised learning now). Instead of using specifically curated and labelled resources to learn from, GPT-3 learns on its own, using essentially every text in existence. Every 1D fanfiction, every conspiracy theory, every misleading Quora post is used for the sole purpose of learning to sound like a human. Beyond this level of efficiency, unsupervised learning has other advantages. One of these is that it bares a great resemblance to how humans learn. Rather than being spoon fed information, humans learn through guided inferences based on our experiences. Massive amounts of unlabelled data give computers the liberty to do the same, letting them represent human behaviours more accurately. Creepy, I know.

Justifying the hype

Ever since its release in May of 2020, discussions about GPT-3 have exploded, garnering much attention. And while this may seem unjustified to some, there may be a reason many swear by its legitimacy.

Many of GPT-3’s fairly recent predecessors, like GPT-2 and Grover were built around the same purpose as GPT-3. For the most part, they could write coherent, well written pieces. Where they differ is that they didn’t really know what they were talking about. Since their learning is also based on human works, everything they write makes grammatical sense. However, the algorithms don’t really digest information they write, and therefore are unable to answer questions, or write factual information pertaining to a topic or question (and oftentimes attempts to answer questions are inaccurate).

On the other hand, GPT-3 can do this with ease through its unsupervised learning process, picking up on word sequences enough to retain facts. I was able to experience this firsthand through playing the game AI Dungeon, one that uses GPT-3 to let users experience limitless fun. By simply picking a theme and the name of my character, I could write a whole story that the computer would participate in. Playing a mystery game? Re-route the story completely by solving it and moving to France, there are no boundaries. Because GPT-3 can recognize and understand each word you input, it’s able to play along, and uses this ability to further the story.

Example playthrough of AI Dungeon. NOTE: The lines with the character next to them represent user inputs.

It’s weaknesses

“The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out.” — Sam Altman, CEO of OpenAI

Rewind, maybe GPT-3’s hype does deserve a little criticism. When discussing its strengths, many analysts are quick to point out its flaws.

Early in its release, many believed GPT-3 could put programmers or publishers out of a job. However, major flaws in GPT-3 prevent it from doing so. First, it doesn’t possess one of the most integral qualities needed to replace people in the programming field, memory. Because its outputs are limited to 2048 linguistic tokens (characters), and because it’s unable to retain previous outputs, they are essentially beyond it. This is relieving news to our massively growing field of programmers, as retention of context for a program (its objective) is necessary. Unfortunately, this isn’t so thrilling when you factor how life changing automated coding could be. For example, many have seen potential in its ability to monitor and regulate greenhouse gas emissions to a massive extent, something that if made systematic, could be one of few things that impacts rising temperatures.

Another highly noted flaw with GPT-3 is the bias it would present if prompted to write about certain topics, usually relating to social or political matters. Unlike (some) humans, it lacks the ability to make sound judgements beyond what it’s taught. Due to its teachings being derived from online media, writings by GPT-3 have shown bias against marginalised groups, or bias towards certain political groups. The establishment of this idea highlights a broader flaw of GPT-3– it’s unreliable. Due to the extremely wide variety of data it consumes to perform our desired tasks, it’s nearly impossible to ensure 100% accuracy, impartiality, or reliability from anything using its algorithm.

Current uses of GPT-3

Despite these troubles, GPT-3 has been used a considerable amount upon its release beyond saving your english grade. Most notably, over 300 apps use it for customer service purposes.

Application Viable gives GPT-3 data from surveys or reviews of an establishment, and identifies common themes in them to provide feedback on behalf of customers. This is made easy for businesses because it works in a question answer format. If asked “What do customers find upsetting about our shopping process?”, it may answer something along the lines of “Customers take issue in the loading time it takes when moving from one page to another whilst shopping. Additionally, they want a way to edit display settings, and see more options at a time.”.

Algolia, another app, essentially does the opposite. Working as a customer service bot, customers can ask questions regarding a website, product, or virtually anything else. Then, they receive a similar answer to the response shown above with more than 91% accuracy, attributed to GPT-3’s ability to grasp more complex concepts than other algorithms.

Ok, and?

What does the development of GPT-3 mean on a more widespread level? While it clearly hasn’t been perfected yet, it’s important to see beyond this to understand the possible weight it can pull on global crises.

But better: InstructGPT

Just shy of two years after the release of GPT-3, open AI developed what is described as the new and improved version: Instruct GPT. InstructGPT and GPT-3 differ in the way they learn, in that InstructGPT uses feedback from users early on to alter future outputs. This means it’s able to produce less offensive language, less misinformation, and less overall mistakes without sacrificing the accuracy of its work.

This rapid evolution, along with GPT-3’s progression relative to its successors, shows how much more is in store for it in the near future. So if it does reach this level of near perfection in the next few years, what does this really mean?

Helping solve a global emergency: The freshwater crisis

The Freshwater Crisis, as the name suggests, concerns a lack of a key necessity for millions around the globe. The World Wildlife Fund predicts that by 2025, roughly two-thirds of the population will lack freshwater. TWO.THIRDS. Therefore, if GPT-3’s brethren permit, this is an issue that requires immediate action.

One of the most promising ventures to resolve this has been a device that uses sensors to detect changes in water quality, which are a driving cause of water-borne illnesses. This poses a problem, however, when you factor how difficult it is to make this a useable technology. The constant need to adapt it to different settings simply wouldn’t make it practical. However, if GPT-3 continues the track it’s moving in, many see automated programming to be a possible reality. In turn, perfecting this water scanning technology could be done with ease, and could cut costs greatly. While it wouldn’t solve the issue in an instant (though I wish it would) it has the potential to be one of the only technologies making a real, tangible difference.

It doesn’t stop there

Though, as noted before, these life changing technologies (via GPT-3) aren’t going to come about immediately. If we don’t pursue a means of mitigating the freshwater crisis in the meantime– we’re essentially screwed. Relying on the progression of GPT-3, one that quite frankly isn’t certain, won’t bring us closer to a solution right now. Thousands of organisations, including The Clean Water Fund and The Global Water Challenge are receiving funding and efforts towards relief of this issue. Sometimes, taking the time to educate yourself on how you can make a change can do worlds more than any emerging technologies of our time.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store