Except it’s fun: Generative AI in 6 minutes
Very late to the party…. I know.
Show of hands if you’ve used Generative AI, such as ChatGPT, Gemini or DeepSeek? 🙋Good, full class (or almost).
Now, keep your hand up if you know how GenAI works? Oh…😅
If you’ve had to put your hand down, don’t feel bad! It’s a complex topic. Your role may well benefit from using the newest* (I say newest.. we’ll see about that) shiny tech, but “I’m not a data scientist, I don’t need to learn how it does what it does!!”. To that audience I say… where’s your sense of curiousity and wonder? 😛
OK, jokes aside. If you’re interested in a quick (no technical expertise required) primer into how GenAI does what it does, join me over the next few minutes! Disclaimer: If you’re instead looking to upskill your use of these technologies (eg: How do I write better prompts?), or interested in the mathematical details, this is not the right blog for that.
Three steps to knowledge:
- How does “AI” work to start with
- What makes “Generative AI” any different
- So — how does GenAI work then
1. How to predict the future 🔮
In a great simplification: Predictive modelling (of which the infamous Machine Learning is a branch) is most commonly used as ‘supervised’ — You pick something to predict, and you try to do so using other, related things you have access to. The “something” is called a target (as it is the goal of your inquiry), and the things you use are called features. (both have other names, that’s besides the point of today).
Oh — And Machine Learning is just a subset of AI (though the two are often wrongly used interchangeably), and so is Generative AI. Here’s a cool blog explaining this all if you’re interested, credit to the author.
Imagine you’re stuck in a meeting, and you have no windows immediately around to answer this for you: How can you predict the weather outside? Well.. usually, just look it up. Though “local” weather reports are not always exact to your location (if you happen to live in England like me, you know how quickly everything changes). But there are other indicators: Temperature, season, how people are dressed and — importantly — what was the weather like 2h ago, when you last went for a coffee? ☕
All of these (and many others) will give you a good indication of what the weather likely is outside. The current weather is your target, and all the indicators mentioned above your features. It won’t always be spot on, but decent enough! In a nutshell, this is Machine Learning:
The study of updating algorithms, fit for inferring future outcomes based on observed data.
Absolutely not the definition, but it captures what we need. OK — So: AI is used to predict, based on related stuff. How is Generative AI, a subtype of AI, different then?
2. What’s special about GenAI 🎢
Nothing. There are many misconceptions surrounding GenAI, so I want to quickly address two:
- How it works is SO different from other AI!
Nope. Still predicts stuff. OK, let’s be pedantic — when “stuff” is collected and stored, we call it data. It uses data, to predict data. (We’ll get into the how next section)
- Generation of unstructured data (images, text, sound etc.) is so new!
It’s a lot better than it was. But new? Depends on how you quantify that I suppose. Ever heard of Eliza? Or perhaps Google’s old (and really funny sounding) chatbot?
But why is it recently indeed that we’re all amazed by how good it is? Two main reasons, one easier to grasp than the other. The first one: Data. The more stuff (now we know, data) you have, the better you’ll be at predicting related issues. We’ve collected SO MUCH data over the years, it’s astonishing.
The second: Context. Remember the days Google Translate was amusing, if not ridiculous at times? Long gone. If you’re more technically versed, the words you’ve been likely itching to see me write are: Attention mechanism.
Imagine the weather example from earlier. It’s been sunny all day long, the news report you’ve seen this morning showed clear skies, and yet… you step outside, and a rain spell out of nowhere! You should’ve expected it. Afterall, you are in London. Unpredictability is the name of the game here, and besides, you’ve seen those nasty clouds hovering hours ago…. ☔
Why is all of this relevant? Because of context. GenAI uses a mechanism for enforcing the importance of context in its surrounding text/image/whatever generation called attention. In short:
The models will pay respect to the most relevant/important aspects of the features and surrounding context.
This, is indeed new. You can read more about the technical deets here, but for our purposes just remember — context & lots of data.
3. OK — but how does it do it? 🌌
Summary of what we know so far:
- GenAI predicts a “target” using data.
- GenAI uses an “attention mechanism” to enforce context.
So, the only question really is: What is the target, and what is the data it uses to predict? The answer, of course, differs depending on what we generate (images? text? videos?). However, everything on this level of data science can be represented as vectors. Whilst we’re not going to bother with what those even are, understand that — if you know how “text” is generated, you can expand further (since it’s all vectors!).
How is text generated then?
By predicting two things:
I. The missing word in a sentence, based on the other words and overall surrounding context.
> Principle: You purposefully hide/mask a part of a sentence, and have your model predict it. What would make sense there? Have I seen similar things before?
This is known as Masked Language Modelling (MLM)
In the example above, without any more context, “flowers” seems most likely. “cars” is a far cry, but “broccoli”, whilst a bit obscure, is possible. This is a large part of the reason text generation holds some degree of arbitrariness to it — there’s no “correct” choice to this MLM problem.
II. The next sentence, based on the ones before and surrounding context.
> Principle: You feed the model 2 options, and have it pick the one that most “logically” follows a given sentence. What would our normal human texts usually look like?
This is known as Next Sentence Prediction (NSP)
In this instance, there should be a clear winner.
And that’s it!
There is so, so, SO much more to learn about GenAI, but you should now, hopefully, feel more confident the next time you use your favourite LLM that you understand how it does its job.
Hope you enjoyed, see you next time. 👋
This blog was NOT written with the help of any Generative AI. Though it would’ve certainly helped =D
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I’m open to answering any other questions in private — just drop me a message on LinkedIn or Email :)
Find me on: Github || LinkedIn or via email at axl_acc@yahoo.ro
