Is ChatGPT just a Search Engine?
First off — having ‘just’ in the title is rather flippant. ChatGPT, Google Bard and their competitors are tremendous achievements, heavily based on academic and community research which everyone in the industry should be very proud of. But at the same time, these systems are fooling people into assuming there’s more intelligence operating behind their output than there really is.
To explore this, let’s have a look into what the fundamental technology really is. At the core of it, these are very sophisticated and impressively trained “Deep Learning” neural networks.
Don’t worry, we won’t be going into any maths on this, but it’s important to cover some of the technical basics.
Deep Learning has been getting a lot of hype for a few years now. Initially because, well, let’s face it — it’s a great name. But there seems to be relatively little understanding of what it actually is, or for that matter even what Artificial Intelligence is.
I’ve spent the past 20+ years of my life working on Machine Learning as well as Search Engines and have had the privilege of working on some of its most ground-breaking uses in the real world, but even I wouldn’t claim to know everything there is to know about AI — sometimes the more you know, the more you realise you don’t know.
Nevertheless, I wanted to offer my own view.
To address what Deep Learning is, I’ll provide an overview of its place in the world of AI, its technical functions, and its real-world capabilities and limitations.
The world of AI
Here’s a hopefully fairly self-explanatory diagram on the matter:
So, Deep Learning is a part of the Artificial Intelligence world. Specifically, it’s part of the Machine Learning world.
More specifically, there are lots of ways of performing machine learning, ranging from clustering algorithms, to support vector machines. They all have their own purposes, strengths and weaknesses, and as with most things you just need to know which is the best tool for the job you need doing.
Deep Learning is spawned from a type of “Supervised Machine Learning” (meaning it needs people to train it) called “Neural Networks”. The “Deep” part of it largely refers to having the type of more complex network architecture which has only been possibly in recent years due to the availability of the computing power required.
Neural Networks consist of a bunch of ‘Neurons’ which don’t really work like the ones in your brain — but they are analogous, in that they all do a very small job. They’re not individually programmed to ‘understand’ where they sit in the bigger picture, they all just do a little job on a little bit of the data, and as the training data flows through the network, all the pieces ‘learn’ about the relationships within the dataset.
Every Neural network consists of ‘layers’ of these neurons that the data flows through, and what the layers do depends largely on what it needs to learn. So those of us working with neural networks spend a lot of our time experimenting with different layer types and sizes to see what gets the best learning results.
What makes a neural network, like the one above, a ‘Deep Learning’ network is simply the number of hidden layers we can get in. The more layers, the ‘deeper’ it is. A few years ago, available computing power simply wasn’t available to handle lots of layers. These days you can have dozens or even hundreds of layers, allowing the network to learn far more detailed relationships within the data.
What is ‘intelligence’ in computing?
Now, although above we’ve explored the technology behind ChatGPT, that hasn’t actually told us what most of us really want to know.
What we really want to know is; what’s it capable of, and what are the limitations?
To answer that, we have to dig a little deeper into the matter of intelligence.
Rather famously, all the way back in 1950, Alan Turing (arguably the father of modern computing) proposed that when a computer could respond to a human in a manner indistinguishable from a human being, it could be deemed to be ‘intelligent’.
Are we at that point? Well, not yet. But we’re probably not far off with chat bots like ChatGPT. It likely won’t be much longer before we cross that line for a relatively brief, non-specialised conversation.
But we have to question whether the Turing Test is an accurate and complete assessment of Artificial Intelligence.
As many people before me have elaborated on, intelligence and how it’s linked to awareness, conceptual understanding, and the deeper nature of biological intelligence, is much more complicated than being able to blurt out a sequence of words in the right order.
A ‘deep learning’ network does just that. Systems like ChatGPT and Google Bard are very sophisticated, without a doubt, but all they do is analyse detailed relationships between words in sentences from training data and then feed you a long list of what the ‘most likely’ words are you are asking for. They don’t ‘understand’, carry any awareness or have any logical or cognitive perspective — that’s purely the domain of the original author of the training data. The structuring of the replies may appear to demonstrate an understanding of the subject-matter on the surface of it, but it’s merely had enough training to teach it what a good reply looks like. That’s why you may notice that the replies you get are very formulaic and ‘middle-of-the-road’.
So, is the Turing test sufficient to determine whether a system can be deemed intelligent?
Not by a long shot.
Yes, you might argue that humans are also just automatons blurting out sequences of words we think the other party wants to hear, and you might be right, but we do so based on a much, much more complex and sophisticated set of metrics. We learn and reason even as we talk, we receive inputs of many types and we understand the context of the original information and sources and are able to differentiate different contexts to the same information to gain differently reasoned insights.
Intelligence isn’t mimicry, it’s abstraction and reasoning.
Now, a big mistake to make would be to claim that reasoning is the sole dominion of humans. Another big mistake would be to say that humans are good at it. We all reason based on our own perception, which is determined by the fairly small number of inputs we receive, and often we reason incorrectly and/or with biases. That’s why it’s relatively easy to mass-manipulate people’s perspectives on social media.
Nevertheless, deep learning networks are still a long way from being able to do any kind of reasoning, extrapolate insights, and for that matter, show actual intelligence. And that’s due to the nature of its architecture. Neural networks are very powerful by design, however they are also limited; it takes thousands upon thousands of pieces of information to learn just a little.
By contrast, human intelligence is able to take small pieces of information, abstract them into conceptual facts and apply those to a wider set of understandings to learn.
For example, if we tell a child that the oven is hot when turned on, the child can apply that single piece of information to all ovens and probably even stoves and kettles, including ones they haven’t ever seen. That’s just not how Neural Networks work; you need to feed them thousands of instances of every type of oven, kettle and stove being hot, together with details on when something is classed as ‘hot’. That’s why a Neural Network can’t learn from a conversation, and why they aren’t actually intelligent.
Systems like ChatGPT get around (part of) the contextual limitation by feeding the whole conversation back into every question, so that the network has access to the questions that came before in a conversation. It’s like reminding it of an entire conversation thread every time you ask a new question. That doesn’t make them intelligent.
Cross-correlation of information, abstract conceptual learning and deductive reasoning are a long way from being possible in Deep Learning Neural Networks. In fact, they may never be capable of it.
So is this ‘intelligence’ or a search engine?
Undoubtedly there will be new Machine Learning architectures which can perform these tasks, and they will be used in conjunction with Deep Learning networks, which will bring about interesting new dynamics to the world of AI, maybe even ones which start to approach the benchmark of ‘intelligence’.
Meanwhile, what we actually have are these very extensive neural networks, which can collate vast amounts of data and regurgitate a summary to a user looking for information. If that’s not a search engine, then what is?
I’ll be exploring the function, purpose and evolution of search engines in another post…
