Dreams in the machine
When I-Robot movie came out, it really put me in a bad place. Being young and a believer in a uniqueness of humanity, it was very difficult thing to come to grasp with a position, where a man-made intelligence acts with free will and not only that, it also dreams. I thought, the greatness of man was that he is intelligent and the idea that something more intelligent can be created by our own hands seemed like an impossibility at the time. Now, I’m not only looking forward to it, but devising plans for it to become reality. What fascinated me with the idea of dreams of a machine is the fact that we ascribe that ability to be quintessentially human. Although, I do believe that animals must be dreaming too, as suggested by an article on national geographic here. Which is why, I thought it would make sense if an AI starts dreaming too. It seems not only far fetched, but also useless to have this kind of conversation, unless we understand why we dream. Now there are many theories why humans dreams, but I am inclined to the idea of dreams being a classroom training for our mind. Here is what I think happens when a person dreams: The brain creates a simulation of real world in our head, disconnects our body from the actions we take in dreams and uses it to send feedbacks inside the dream. Our mind is incapable (most of the time) of understanding that it is a dream and deals with it as if the events are happening for real. Most of the time the dreams are very exciting and our responses to the events happening in the dream are also extremely intense, as in the case of nightmares. This process allows our mind to go through a particular experience that it hasn’t gone through before or may have to go through in the future. The past too is repeated and mostly, with variations. If we think about it, we can see that this is how the brain teaches itself to handle different situations that it needs to prepare for. There must be a dream generator function in our head that creates a scenario which is run through in our dreams and our mind learns to deal with that situation. So, it is a kind of a learning system, that teaches our brain to explore the possibilities outside our everyday experience.
What has this got to do with machines?
I believe machines learn via dreams too, only we do not call it such. We call the simulations. The designer of simulation creates a scenario and the machine is run through that scenario, which allows us to train the machine for different eventualities without having to incur the cost of training the real thing in the real world. In the world of machine learning, simulations happen with training of the algorithms with data. This training data is collected from the real world and then passed to the algorithms, which will generate the world-model based on the data and then we can test the algorithm with another set of test data for its accuracy. These systems are not absolutely accurate and the failure rate goes down when the training is properly balanced between under and over training. Even in case of reinforcement learning systems, the system learns by rewards and punishments based on the moves it makes by either exploiting the known best path or by exploring to figure out even better baths and updating its parameters or world-model. It is also understood and accepted that, the data keeps changing and therefore, there is a need to constantly learn and use some feedback mechanism to learn continuously. This is very much similar to way organic beings like us learn too. However, in the above approach to continuous learning, there is a limitation that comes due to the feedbacks being generated from the data already seen. Wouldn’t it be interesting for machines to also try some data that it has not seen? Much like the way we dream, where we are jumping off of a cliff and flying or walking in pyjamas/naked in office or school?
Dreaming away to glory!
The proposal I’m making with this post is to design the learning components in such a way that there would be two phase of its working:
- Production mode: Production mode is normal operations where the machine is given the real world data and its output is consumed by the real world. The feedback here generated would be directly fed in to the system while its online.
- Dream/Simulation mode: In this mode, there is a need for world generator engine (world here represents the environment of the machine [function/algorithm]) with which the machine is suppose to interact. The world generator is what simulators today work as, however, with the slight change that it is suppose to come up with new and interesting scenarios that may or may not occur generally in real world. This would allow the algorithm to learn something it has never seen before and would not see possibly for a long time.
The importance of simulator / world generator
The biggest takeaway from this idea is the concept of world generator to generate simulation scenarios. To understand it, imagine that a game automatically creates the levels of the game by studying and understanding the existing levels (meeting with all the constraints such as level must have a way to finish successfully and failure should be based on the scale of difficulty). This ability to generate levels in game would cause the games to take exponentially long time to end and possibly make them very boring. However, for a machine, this routine game play with tiny to huge amount of variations will end up making the machine intelligence very strong. A good use case in computer science, which may speed up the idea of such generators, is the concept of generating test cases. There are existing frameworks and service providers that allow randomizing the data for testing, to cover the corner cases while testing a given service. These can be taken further by allowing the generator to study the existing cases and coming up with many new variations of the existing test cases, while ensuring consistency with the constraints of the system. This will not only allow the developers to capture the bugs early, but also free up a lot of time developers spend writing those test cases.
In conclusion, this blog post is just a way to throw the idea out there and develop it over time. I hope I will be able to inspire a new startup with this post. May be, I might be the one to start it up! It would be fun to have machines with the dreams and then another sci-fi movie like I-Robot will leave the genre of fantasy and enter reality. Cheers!
Originally published at www.3371aspectz.co on April 13, 2017.