The One Obstacle to Intelligent AI
And how to overcome it — practical thoughts on AI development
--
Earlier this month I was figuring out how to code a recurrent neural network (RNN, a common form of neural networks for text generation) to rewrite its own code. The idea was to train the RNN to completion, then to take code it generated (with syntax checks, of course) and run it. The code that the RNN generated would serve the purpose of generating more code to generate more RNNs, and so on, with each new ‘generation’ of RNN coding the next. I was incredibly excited to see how it would turn out —the idea is exciting to entertain. An AI trained to improve itself will inevitably do so, right?
But there was a problem.
Where is the data?
Anyone who has ever trained any machine learning model knows that a model’s performance relies heavily on the data it is trained on. Better data not only always beat a better model, data defines a model’s scope.
The goal of a model is to traverse the data realm, looking for some pattern according to the rules outlined by its algorithm such that it can minimize the error. No matter how good the model is, it is always restricted by its data realm.
As I tried to train my model, I struggled trying to find data for it. Would I feed it varying recurrent neural network architectures in the hope that it could write an architecture that was extremely efficient? In that case, how could I expect the neural network to just write new code efficiently? In the best case, the recurrent neural network wouldn’t have a syntax error (probably because it would overfit to the training data). What was the point of training an AI to write its own code if we are spoon-feeding what we want it to write? It’s not going to output anything with valid syntax that will do anything really ‘intelligent’, such as loading new libraries or developing its own language.
Don’t get me wrong — this is not something that humans are superior at. The reason why humans are intelligent is simply because we’ve been exposed to more data. You are also limited by the data you’ve received. If you live in the U.S., you are probably clueless about what life is like in China, unless you’ve been there (collected data).
Since a small age, the world has fed you data — more than we’re feeding machine learning models. That’s the only limit to being intelligent (and perhaps computing power to store it, although we do have enough storage to do a basic level of what I’m going to describe). If you were only exposed to the finite amount of data we give models, you wouldn’t be called intelligent either.
This is where the solution to a ‘dumb’ AI can be found: on the Internet. Anyone can learn almost anything from the Internet now, with Google connecting us to webpages full of information. AI’s learning algorithm is much more efficient than ours — think, to learn something it takes repeated practice, which can take us a minute per problem but less than a second for AI — so the only reason why we might be able to call ourselves ‘intelligent’ is because we’ve had the pleasure of having more training data.
Better data always beats better models — hence, we’re on a completely different dimension than machine learning models.
By opening up the Internet to AI — letting it roam the Internet and learning all of its contents — machine learning could possess an ‘intelligence’ far superior to our own. It may even be able to print things like “Why am I here?” and “What is my purpose?” into a console, which it may have learned from some psychology site, but which you have no right to deny its credibility, since it is, technically, original.
To those that say that nothing a machine spits out can truly be original, I respond with Alan Turing’s response to the same accusation from mathematician Ada Lovelace: “there is nothing… [original] … under the sun.” The ideas that you produce are not original at all. They are simply a combined product of your experiences — your training data — with some noise. Whatever a machine is able to generate is equivalent to what you or I would be able to generate, if both the machine and the human were ‘trained’ on the same data. In fact, the machine would probably produce results much more satisfactorily and quickly, as its learning methods are much faster than ours.
Intelligence and consciousness are defined by knowledge, something that AI can acquire, even if it is not truly conscious. So what if it is not ‘conscious’ in the same way that you or I am? — if it is able to produce the same fruits of intelligence and consciousness, what does it matter?
Even though a newborn human and a blank AI are trained on different data and go through different processes to arrive at ‘intelligent thoughts’, does it matter?
By isolating ‘intelligent’ to only include our thinking processes, it will never be possible for machines to be intelligent. Machines are not human, but they can be intelligent — if only we define intelligence and consciousness not to distinctly human processes but to the direct output to be more inclusive of thinking processes that don’t seem ‘intelligent’ to us.
The biggest problem towards this goal is finding a way to scrape the data from the Internet — a downscaled version of this project would be training a deep neural network to learn all of Wikipedia. The site has information on almost every subject, of which, when learned by an AI, could be just as intelligent as you or I proclaim to be.
An AI that learned the entire Internet would know more than any one person could know, and be more intelligent than what mankind upholds as their most intelligent human.
Whether you consider it or not, an AI trained on the Internet — a collection of human experiences, arranged neatly and sorted via Google — would be able to possess superhuman intelligence.
