Rog’s Day 4: Imagination, Present
Look at an object in front of you. Now keep on looking at it directly while imagining in a different color, purple for instance.
Our common experience is that although we can imagine the object in purple, we still see it in its original color. The imagined never takes over. Somehow, the senses are always stronger than our imagination [coincidentally, the ink on the other side of this page is really distracting].
But what if you could make your imagination stronger than what you felt?
You would be catapulted into a world in which anything you want is available. Any desire can be fulfilled. No boundaries exist. Anything.
It’s a compelling and difficult idea. Compelling because anything you can imagine would be felt as if it is actually there, projecting you into limitless worlds. Difficult because reality wouldn’t be interesting anymore — with all the caveats of the case, such as creating a sufficiently colorful imagination in the first place and a broken balance of incentives for survival. It’s a rabbit hole with profound ramifications into virtual reality, psychology, the human condition, society, and much more.
Let’s assume that it is somehow possible to make our imagination stronger than what we feel. We might have indications that this would indeed be the case through drugs, medication, surgery or injury. But let’s take a step back first: What does “stronger” mean? Is it literally a signal with a higher intensity or firing rates? Or is it a specific temporal pattern? Or does imagination use altogether different pathways than sensory signals?
Neuroscience showed that when we imagine something, the same areas are activated as when we actually sense it. We imagine seeing something, and the visual cortex lights up. We remember the smell of a lover, and our brain catches fire. Maybe not actually, but it kinda feels that way. Maybe it was a loose association that triggered the memory. Or maybe we were thinking in hypotheticals, “what if?” These all send signals down to the sensory areas. From where then they move up again, in a wide and long loop.
If we apply the principle of parsimony — which in evolutionary terms appears wise — , the system would prefer relying on the same circuits for both imagination and sensation. Growing circuits that achieve very similarly the same functions but separately appears rather complex. Additionally, there isn’t any evidence for “imagination circuits”.
Let’s assume then, that imagination uses exactly the same circuitry as sensation. Even though the exact mechanisms remain unclear, the hypothesis is compelling: We are able to recreate sensations by stimulating our sensory areas, the very same areas that we use to process data from our senses. Then the induced stimulus is further processed almost as if it were real.
Using the same circuits to process hypothetical stimuli is a powerful paradigm. For one, such a “biplanar” brain would allow us to easily plan and imagine outcomes. There would for instance be no need to explicitly check if something makes sense or not: Thoughts would never be too far from reality without feeling strange. Like a unicorn 🦄 riding a rainbow 🌈. We can easily imagine it, but it doesn’t feel real as a horse does.
On the other hand, let’s say I want to imagine something realistic, like taking a different route to go home, one that I never took before. If I use the same circuits that I use for sensing and processing my way home and its surrounding, imagining a different route would elicit similar associations, and have cues related to the area.
Simulating the sensation
The intuition feels good, let’s see what it gives in simulation. We created a small mobile robot that has just 3 little distance sensors and 2 wheels. Using its sensors, it learned how to avoid obstacles. Please mind that the environment is based on a stochastic field, taking care of injecting a healthy dose of randomness, and the subsequent need for statistical approaches in the behaving systems.
We then added a sort of hippocampus to its brain, allowing it to locate itself and the position of the obstacles. The robot has no specific purpose for now, but it has the capability of imagining individual scenarios, all while maintaining its default behavior of not hitting any walls.
At first, little Bloom — the name we gave the wandering and imagining robot — would only be able to imagine in a straight line (light blue line), one direction at the time. As a result, it would imagine going through walls and out of its tiny robot world.
However, as soon as the obstacles were detected once and thus became part of the hippocampal map, the imagination became more concrete, and the obstacles minded.
It’s fun to see Mr. Bloom in action, and how different initial plans create different scenarios. Technically, the imagination was coded with a simple flag:
spoon = false means that it’s real,
spoon = true means that it’s imagined.
Now, we thought it’d be nice to give Mr. Bloom a purpose, or at least a compelling need to plan. That turned out disarmingly simple: We just attached a cost function: The less obstacle encountered per distance unit, the better. Then we would place the best option as a set of instructions in a short-term memory. Does Mr. Bloom find the best paths? No, but they appear to be reasonable.
In our first iteration, we used mathematical formulas to calculate paths. In the second, we used a shallow neural network, and the resulting behavior was the same.
Learning by Doing
So, what does this mean? It means that Bloom is able to “imagine” outcomes that are consistent with its reality, without having to explicitly program it. Moreover, this kind of “imagination” only started to be useful when there were memories.
Because the same circuits were being used, despite being faster because there was no actual motion involved, the “ imagination” took away computing cycles from sensing and reacting in reality. By modifying the parameters, the robot would be able to plan ahead over a longer period, but only with the trade-off of being less agile in coping with reality. Slightly uncanny.
Of course, the lines between a robot’s “imagination”, “simulation”, and mere electrical signals are very fine. Semantically heavy, but almost invisible. Although we like the concept of a simulated robot simulating what could happen, we’re going to stick with the concept of the robot’s imagination, because our ultimate goal is creating robot consciousness, and we need to be able to name it when we see it. It’s still a long journey, but concepts are becoming clearer.
