Research on Effect Models: Exploring Life and Intelligence
When developing complex systems using software, a task called modeling is performed.
A model refers to a prototype or template. For example, when designing a building, there is a process of creating a small-scale model. This allows us to capture the overall image of the building.
In system development modeling, we organize the information necessary for the system and the relationships between that information, and consider changes in these information and relationships.
For example, in a company’s personnel system, information about a list of employees and departments is necessary. There are relationships such as the hierarchy between employees and departments, the belonging of employees to departments, and the hierarchical structure between departments. And changes in information, such as employees joining or leaving the company, department transfers, and organizational restructuring, can be considered.
I am conducting personal research on the origins of life and the nature of intelligence. In this article, I would like to think about an effect model as an abstract model of these.
The effect model is a model that represents multiple elements influencing each other. It holds a list of elements as information and the relationship that these elements influence each other. Over time, elements change as they influence each other.
Compared to the personnel system example mentioned earlier, this model may be hard to visualize due to its high level of abstraction. On the other hand, its high level of abstraction makes the model simple.
This effect model is an abstract model of an algorithm called a cellular automaton, which behaves like life and is called a Game of Life. Also, among neural networks used in artificial intelligence, those with a loop structure can also be called effect models.
In other words, the effect model could potentially be a model for overlooking one aspect of phenomena such as life and intelligence. Of course, life and intelligence are very complex, and it is impossible to fully grasp everything with one model, but by considering such simple abstract models from various perspectives, we can gradually capture the overall picture.
Sorry for the long introduction. Below, I will discuss the effect model in more detail.
Effect Model
I would like to model a system where elements influence each other as an effect system. This is called an effect model.
The foundation of the effect model is elements and influence. Influence can change the structure of elements and the way they influence and are influenced.
However, incorporating changes in structure and influence makes the model complex.
Therefore, let’s start by considering the case where elements and influences are fixed.
Imagine a structure where elements are lined up in a row. One element influences multiple elements. And one element is influenced by multiple elements.
This simple effect model can be represented as a special kind of recurrent neural network, where only one layer exists, and the output of that layer becomes the input to itself. It can be called a single-layer recurrent neural network. Recurrent means it has a loop structure.
If you fix the parameters of this neural network, give an initial state, and repeat calculations, the influences will ripple through each other and change into something complex.
This is also a cellular automaton. In cellular automata, adjacent cells influence each other. This can be replicated in the special neural network mentioned earlier by setting the parameters so that only adjacent neurons influence each other.
Also, the rules of cellular automata determine the output of a cell based on whether the total value of adjacent cells is a certain value. This can also be replicated by setting the output function of the neural network according to the rules of cellular automata.
Therefore, both cellular automata and single-layer recurrent neural networks can be said to have the properties of a simple effect model.
Behavior of the Effect Model
Cellular automata, as known in the Game of Life, are known to undergo complex changes reminiscent of life.
Since a single-layer recurrent neural network encompasses the properties of cellular automata, it undergoes similar complex changes.
Below, Figure 1 shows the temporal changes in a cellular automaton, and Figure 2 shows the temporal changes in a single-layer recurrent neural network. There are five planes, showing changes over time from top to bottom.
In a cellular automaton, the next state of a cell is determined by the influence of adjacent cells, so changes occur locally.
On the other hand, in a single-layer recurrent neural network, the next state of a neuron is determined by the influence of all previous neurons. Therefore, unlike cellular automata, there are no local changes. Instead, the whole system changes dynamically.
On the linked page, you can observe these temporal changes. By pressing the Reset button, the initial state changes randomly, and by pressing the Start button, temporal changes begin.
a) General Cellular Automaton
b) Single-Layer Recurrent Neural Network
General cellular automata can be easily found on the internet with their changing rules.
However, the single-layer recurrent neural network without input and output shown here is my original creation. Basically, it is based on the concept of neural networks, with randomly set parameters for weights and thresholds. On the aforementioned site, it is created with JavaScript, so those interested can check the processing content and parameters.
Dreaming System
The cellular automata and single-layer recurrent neural networks presented here, without external input and output, internally propagate influences and undergo complex state changes. This can be seen as akin to dreaming.
The human brain also generates complex images internally, without any external input or output.
The Phenomenon of Dreaming
I believe that when humans dream, it’s not that the brain enters a special mode, but rather that the nerves for input and output are simply closed off.
Dreams are often absurd and include events that would not occur in reality, like returning to childhood. Hence, it could be thought that the brain enters a different mode from when it is awake.
However, I think that just by closing off the input and output nerves, our brains can imagine things that don’t exist in reality, or don’t question being in a different age or role than in reality.
There’s a known phenomenon where generative AI fabricates information that doesn’t exist. This leads to discussions that while humans can ground information in reality, generative AI needs additional mechanisms to do so.
My view is that humans, when asleep, are unable to perform this grounding.
Grounding in Stories and the Real World
Conversely, when awake, the human brain could constantly be conjuring absurd, dream-like thoughts. However, information entering through our nerves quickly dispels these dreams. This is the effect of grounding.
This understanding explains why we get immersed in the world of novels or movies. From text or fragmented visuals, we imagine a world in our minds like we do in dreams, feeling as though we are inhabitants of that world.
This immersion is broken by stimuli from reality, like noise or physical pain. The world we imagine in our minds is fragile and easily shattered by physical stimuli. This is also a form of grounding.
Furthermore, we lose interest when a novel or movie has an implausible plot or a world that collapses upon itself. If the discrepancy between the story’s settings and our mental image is too large, the world we imagine collapses.
Dreams, being self-created, are less prone to this self-destruction.
Also, the real world is, to the brain, much like a novel or movie. Based on fragmentary images and stories heard from others, we imagine the real world.
Since there’s no stronger grounding than the real world, it’s less likely to experience the disengagement seen in waking from dreams or losing interest in a story.
Grounding in the Effect Model
Returning to the original topic, cellular automata and single-layer recurrent neural networks can be seen as simple prototypes of the phenomenon of dreaming.
This means an effect model without external input and output has the mechanism of a dream. Connecting it with external input and output would then provide the effect of grounding.
In other words, the human brain is also an effect model.
We learn about the real world and imagine it in our minds based on fragmentary information. This ability to imagine is merely dreaming when asleep, but becomes speculation about the real world when awake.
Thanks to this speculation, we can walk believing there’s ground or stairs based on a glance, without having to stare intently. This is likely a speculation based on pattern recognition from experience.
We can walk through crowds without bumping into each other due to speculation based on simulation. While recognizing the shape of the ground or stairs is pattern recognition, how we will walk on them is speculated through simulation.
Our ability to act without meticulously checking every detail of the real world is thanks to this speculative ability, a combination of pattern recognition and simulation.
I believe this ability to simulate is due to the mechanism of dreaming and is carried out by the effect model.
If the brain is an effect model, it must be able to mimic the elements and influences of the real world. That’s learning. We learn the elements and influences of the real world and build an effect model in our brain that can replicate them.
We use this effect model to automatically imagine the real world, constantly speculating. And when we fall asleep, we continue to speculate, but without the grounding of input, even absurd speculations proceed unchecked.
This state can be seen as an effect model without input and output, undergoing complex automatic changes as shown in Figures 1 and 2 above.
In Conclusion
As I wrote in the beginning, the cellular automata and single-layer recurrent neural networks presented here are very simple effect models. Although simple, they represent the essential properties of an effect model. When considering advanced effect models, these simple ones serve as a starting point.
For advanced effect models, first, we can consider models where the parameters that influence change. In neural networks used in artificial intelligence, the parameters that indicate the influence between neurons change through a process called learning.
Next, we can consider multi-layering. This is also a technique used in neural networks in artificial intelligence. With a multi-layered structure where influences loop through multiple layers, we should see more complex state changes.
Based on these, we can consider grounding. This involves advancing from an effect model without input to one that includes input. It’s not just about undergoing state changes internally like dreaming, but also about changing parameters so that state changes occur in line with external input. In other words, it involves learning to predict the information being inputted.
Further sophistication is possible by adding the functionality of output. It’s not just about passively receiving input from the external world, but also about influencing the external world through output from the effect model. This means that the effect model and the external world together form a larger nested structure of effect models.
In this larger world, there may exist other effect models with similar mechanisms. It’s a scenario where effect models influence each other.
This reminds us of ecosystems among living organisms or societies of intelligent humans. Therefore, the sophistication of effect models could lead to modeling life and intelligence and even extend to modeling ecosystems and societies.
