How to explain “Generative Models” to a 9-year-old
What the “G” in ChatGPT Means
The “G”, “P” and “T” in ChatGPT stand for “Generative”, “Pre-Trained” and “Transformer” respectively. The real power comes from the “T” or “Transformer”. But, for me, the “G” or “Generative” is the more foundational idea.
In this article, I try to explain what “Generative” means in the context of Artificial Intelligence and Machine Learning, as I would to a 9-year-old.
[Obviously, “Generative” has a formal and mathematical meaning — as does its counterpart “Discriminative”. I won’t go into these here, but if you’re interested, Wikipedia has a decent explanation.]
Two Games
Let’s suppose you are a 9-year-old.
Suppose we play the following two guessing games.
- Game 1 “Guess the tune”: I play a short tune on the piano, and you guess what it is. For example, I play the tune of “Mary had a little lamb”, and you guess it.
- Game 2 “Play the tune”: I name a tune, and you play it on the piano. For example, I say, “Mary had a little lamb” and you play it.
Now, suppose you are very good at Game 1 (“Guess the tune”). Does that naturally imply that you are also good at Game 2 (“Play the tune”)?
No. There are many people who will be very good at guessing the names of tunes, but can’t play them on a piano, or even hum or whistle them.
The converse is also true. You could be good at playing any tune that is named but might not be able to name a tune that is played to you.
Many Machine Learning models are like people good at Game 1 or Game 2, but not both.
For example, you can train a model that can guess the name of any tune played (e.g., Shazam), and you can also train a model that can play any tune named (Spotify is not exactly a Machine Learning model, but that is what it does).
The ability to be good at either Game 1 or else Game 2, is sometimes referred to as a Discriminative ability. In other words, the person or the model can discriminate between tunes or names.
People Good at Both Games
Now, there are people who are good at both Game 1 and Game 2. They can both play a tune named, and name a tune played.
People who are good at both games are often able to do two things that “Discriminative” people (those who are good at only one game) cannot do.
- Given a made-up name of a song (say, “Dhara had a little Frog”) they can make up an entirely new song to match the name.
- Conversely, if you make-up a tune, they can come up with a plausible name for it.
They can do the above because the seem to be able to expand on or generalize or generate on the knowledge of names and tunes that they have.
This ability is known as Generative.
Generative Models, like ChatGPT or DALL.E have this exact same ability.
For example, you can say “Dhara had a little Frog” to DALL.E and it can come up with an entirely new picture that matches that title. That is a principle reason why they are so powerful and increasingly popular.
So that is what a Generative Model is and does.
Hope my 9-year-old and older readers understood!
