Introduction to Learning Agents
There is a certain mysticism around the word “intelligence” such that when computer scientists develop algorithms that learn from input, it is not called intelligence but Artificial Intelligence. My intent is not to tread on this word so instead I will coin a new term that encompasses both intelligence and artificial intelligence.
A learning agent uses sensory input from its environment to adapt its actions over time based on a reward function.
Given these simple requirements, learning agents include cats, dogs, humans, computer algorithms, and even evolution.
The Original Learning Agent
Evolution is the first learning agent. It has one simple purpose: to create life. Like a virus, it has infected all of its hosts with a desire to fulfill its purpose. By optimizing simple organisms, our entire ecosystem has been created. Charles Darwin calls this optimization technique “survival of the fittest”. As a result, the biological world is full of organisms that have their own learning behavior.
Claims about Learning Agents
The purpose of defining something so I can is to be able to ask questions about a learning agent and generalize this across the set of all learning agents.
1) Learning Agents can be nested within other learning agents.
2) In a nested system, the nested agent is slower learning than the it’s parent agent.
3) Accuracy of a learning agent is a function of variance and time
The neural network is well-known a knowledge storing structure of a learning agent. The fundamental building block of a neural network is a perceptron.
A perceptron has a weighted vector, an input vector, and through the use of a transfer function, produces an output of 1 or 0.
If we re-word the question, we come up with: Can a learning agent be a perceptron in the “brain” another learning agent? Evolution is certainly an example of this since through reproduction, we transmit a binary signal down the network.
Inputs are equivalent to genes and weighted vector equates to environmental conditions.
We can also say that evolution is a slower acting intelligence than its perceptrons.
Types of Learning Agents
A learning agent that has been in an environment with low variance can be called a specialized agent. A specialized agent has low elasticity and high accuracy.
Given the same number of training data points, a nonspecialized agent is characterized by high adaptivity and low accuracy. Nonspecialized agents have been exposed to an environment with high variance.
Some Rules for Learning Agents
These are some unproved rules that I find intuitive and self-evident. Q-Learning gives us a powerful framework to think about learning agents because we can change an agents ε (Greediness) and α (learning rate), and see what happens to the complexity of a learning agent.
Rule of Speciality
Increased speciality decreases the ability of the learning agent to adapt to a new environment.
Rule of Complexity
The complexity of a learning agent increases the number of skills needed to adapt to an environment.
Rule of Accuracy
As the complexity of a learning agent goes up, the accuracy of that learning agent will go down.
Rules of Elasticity
- A learning agent can change its greediness (risk) to achieve a higher reward or make a reward more consistent.
- Some learning agents can change the reward itself to return a higher reward or make a reward more consistent.
Rule of Limited Complexity
A learning agent will adapt to the minimum complexity to acheive maximum reward.
*If you give a learning agent something for nothing, this will reduce the complexity of this learning agent.
I believe these rules reflect a truth in ourselves and our environment. Anyone studying biological intelligent systems will tell you that there are many more things we don’t know that what we do. For this reason, I don’t consider it a waste of time to put down what seem to be obvious rules of learning agents. It’s not a complete list by any means, especially since there are no rules involving actions, but it’s a path for starting to understand how to engineer intelligence and to predict the way intelligence will progress.
There’s so much we don’t know about the singularity but one thing we do know is that it will have to follow the same set of rules that every other learning agent. A list of these rules would give us a framework to think about such things before they happen.
A stable list of rules will involve large amount of empirical test data from biological and silicon based learning agents.
I believe we should take biologically inspired techniques very seriously in computer science. One such technique is male/female reproduction. Both male and females have complimentary attributes that work together and maximize the likelihood of life continuance. While self-replication and mutation has been explored in computer science, reproduction among agents that model gender has not been explored as far as I know. Since gender is so pervasive in nature, it must have some optimization advantage too.
We have to keep our minds open to the possibility that evolution is not simply a technique of optimizing an organism to exploit resources but an intelligence, that changes the way it functions over time to maximize the sum of all life creation. We have to consider the possibility that evolution can change its rate of mutation or greediness over time.