GPT-3: The Largest Leap in AI so far?
It was in the year 1989 that Yann LeCun and other researchers at AT&T Bell Labs successfully applied a backpropagation algorithm to a multi-layer neural network, recognizing handwritten ZIP codes. Given the hardware limitations at the time, it took about 3 days (!) to train the network. AI has grown tremendously, in leaps and bounds since then, with numerous milestones along the way, such as the first computer to beat the world champion at chess, Deep Blue, coming out in 1997; the assembly of ImageNet, the largest collection of labelled training images for neural networks back then, in 2007; Google Research’s BERT model with unforeseen capabilities and remarkable performances in 2018, and several others.
In July 2020, there was another such moment when OpenAI released the GPT-3 model, which has taken the Natural Language Processing world by storm, with its breathtaking capabilities. GPT-3 is truly versatile: it can answer questions, produce text, poetry, design, do translations, and even code! This gigantic model, with over 175 billion parameters, might have hit a turning point in NLP, where AI models start to become virtually indistinguishable from humans in language comprehension and generation.
Let’s have a look at some of the truly incredulous tasks that the model is able to perform:
The first application using this tech is an application called Figma- the developers have built an extension to the app which enables users to enter a description of what they want for an application, and remarkably, the model is able to build an app all by itself that matches the description.
An app very similar to the popular social media app Instagram is created based by GPT-3 based on the description, replete with scrolling and interactive functionality.
The next example is a automatic code generation where the model build a basic to-do list application.
The complete code has all the events and triggers for handling the interactions of the application, and it’s all generated on the fly by artificial intelligence! It’s just amazing what can be accomplished by GPT-3, in both the design and development aspects.
Another example of the model is in Excel, where we give it a table with headers containing the labels Company, Ticker, and Year Founded. Quite incredibly, the model fills in the entire table by itself, by estimating what info we require in the table. A blank table with just the headers is given. We ask GPT-3 to populate the table for us, with no additional information provided.
The model fills up the table with companies it thinks are most relevant, along with their tickers and founding years.
The model can also be provided examples from which it can deduce the kind of information needed. If details for two companies are provided in the reverse order of founding dates, the model recognizes the pattern and fills up the rest of the companies following the order!
Another massively useful application of GPT-3 is in automatic text summarization.
Using OpenAI’s API, GPT-3 was prompted with a technical description of quantum computing, and asked to summarize for a second-grader: “Quantum computing is the use of quantum-mechanical phenomena such as superposition and entanglement to perform computation… (truncated the rest)”
GPT-3 returned:
“Some people think that if you had one hundred people working on one problem for one hundred years you could not figure out how to solve it with a regular computer but if you had one hundred people working on one problem for one hundred years using a special kind of computer called a Quantum Computer you could figure out how to solve it.”
When GPT-3 was asked to summarize it for an eighth grader, it returned a more complicated by still lay description:
“When we say quantum computing, we mean computing based on quantum physics. With classical computing we perform calculations one bit at a time. With quantum computing we try to take advantage of a superposition between 1 and 0. This is what we mean by qubits. This allows us to make connections between pairs of bits (qubits) using quantum logic gates. Quantum computing allows us to perform computations faster than classical computing.”
The API can not only summarize, it can summarize appropriately for a particular audience!
The next example is within Wikipedia.
If you’ve ever wanted to seek information from a long article without spending the time reading it all, this browser plugin powered by GPT-3 can help. You type in the query after clicking on the plugin while on a Wikipedia page, and the model searches for the answer on the page.
Because of the vast amounts of information in GPT-3, you can almost ask it anything, essentially using it as a search engine. Google, Apple and other big tech companies have thousands of engineers working on a very similar problem, and this is solved with the Deep Learning model almost instantly.
AI, especially Natural Language Processing (NLP), is evolving at an unprecedented rate. Gone are the days when hardware limitations would put a bound on training models. OpenAI’s GPT-3 is sure to prove to be a big boost to not only the general populace but AI researchers as well, in the quest for an omnipotent model akin to the human brain!
