NeurrNet: Idealization of Sentient Networks in Neuro-Cognitive Modeling
A thought wrapped around the mathematical landscape
The key to a world where machines not only understand, but also empathize, where the boundaries between the human mind and artificial intelligence blur into a mesmerizing dance of cognition and code, might just lie in the promise of a mathematical model of the human brain. The study of neuroscience and trying to model the functions of the mammalian brain is far from recent. In fact, Nobel Prize laureate’s A.H. Hodgkin and A.F. Huxley’s study constituted a model, now recognized as the Hodgkin-Huxley model of the ionic mechanism governing the initiation and transmission of action potentials, dating as far back as 1952. One of the key equations in the model is the membrane current equation, which includes components for the sodium and potassium currents, which remains one of the foundational pillars of computational neuroscience due to its significant contributions to our understanding of neuronal activity (Equation 1). Other models such as the Fitzhugh-Nagumo Equations later led to a novel approach to the problem, paving the way for the study of neural fields.
Since then, our exploration has extended far and wide, delving simultaneously into the realms of computational neuroscience and the pure domain of mathematical cognitive modeling. A recent study goes so far as to apply advancements in cognitive modeling to leverage the potential of incorporating cognitive-inspired mechanisms into the design and control of prosthetic grasping systems, thereby enhancing their functionality and usability for individuals with limb disabilities.
But to comprehend even a fraction of the concept of sentience, we must first scrutinize the fundamental aspects. What constitutes a “thought” in the human mind, and how can it be effectively translated into mathematical terms within the realm of neuro-cognitive modeling?
I’ve taken the liberty to oversimplify the mathematical model (for now), and my definition of a thought based on existing foundational pillars and concepts for the purpose of this article. In this context, thoughts are often represented as ‘patterns of neural activity.’ In a physiological context however, they frequently correlate with the activation of specific brain regions, stemming from complex interactions among neurons. But, as far as we’re concerned in the realms of statistical analysis and decision theory, let’s take a step back and define a thought Ti as a dynamic neural activation pattern within a cognitive network, where each element of the pattern corresponds to the firing rate of individual neurons or neuron clusters associated with specific cognitive functions.
Here’s my thought (one out of many in the cognitive space I’m sure)!
For a thought Ti, let F(Ti) be the cognitive feature mapping function that transforms Ti into a multidimensional vector in a cognitive feature space. If fj(Ti) represents the cognitive feature j associated with the thought Ti, incorporating neural activation patterns, semantic embeddings (semantic associations and meanings), and contextual information (contextual relevance and situational dependencies) derived from the neuro-cognitive network, the function F(Ti) is defined by Equation 2.
I won’t be sticking around too long in the math I promise!
But before we delve into the physical significance, there’s one more term that we must address, much to my chagrin.
A thought prior P(Ti), can be defined using Bayesian inference principles and neural dynamics, incorporating the probabilistic dependencies and interactions between thoughts within the cognitive network. Let’s assume that P(Ti|θ) represents the conditional probability distribution of thought Ti given the model parameters θ, and P(θ) denotes the prior distribution over the model parameters. The integral is taken over the parameter space Ω, capturing the collective influence of neural dynamics, cognitive interactions, and prior beliefs on the emergence and propagation of the thought Ti within the neuro-cognitive context. One can express the oversimplified thought prior P(Ti) as described in Equation 3.
For the math enthusiasts, here’s the in-depth study regarding Bayesian cognitive modeling.
As I said, the math behind the cognitive landscape is moot without it being associated to a broader landscape. For us, this implies comprehending the pathway towards achieving a more profound sense of sentience.
Let’s just imagine the cognitive space (encompassing the latent feature space learned by the network) of a network nicknamed NeurrNet at the moment. NeurrNet is a promise built upon several principles, recognizing the ever-expanding structure of a thought-line. But before we delve into what NeurrNet would require as an emotionally intelligent system, a discrete approach to implementing this system would initially necessitate a fundamental design choice. Should we model the network as a ‘dissection project,’ so to speak, with the aim of emulating the functions of the limbic system, pituitary, cerebrum and beyond, all in pursuit of emotional intelligence?
The obvious answer is that it wouldn’t even come close to being sufficient, for fairly obvious reasons.
Human consciousness, subjective experiences, and higher-order cognitive functions are far more intricate than what current computational and neuroscientific methodologies can fully capture. Despite the strides in artificial intelligence and neural network modeling, achieving genuine sentience and consciousness demands a depth of self-awareness, introspection, and abstract reasoning that surpasses our current technological boundaries (as you can see, this article is quickly taking shape as the ideation exercise I intended it to be).
But does that mean we should halt our pursuit of emotional intelligence? Certainly not, and to my excitement, neither have any of the researchers involved in the field! But what it does mean is that we must confine our study to the modeling of a ‘solitary thought’ and its journey within the cognitive landscape. Our study, under various constraints of modeling, would focus on the concepts of (to mention a few):
1. Thought States and Transitions.
2. Mood Modeling.
3. Fuzzy Logic and Probabilistic Transitions.
4. Memory and Forgetting, seamlessly transitioning into Cognitive Load and Information Processing.
5. Temporal Modeling (Understanding Thought Dynamics Over Time).
Our models exhibit several commonalities that can be harnessed through the wealth of mathematical prowess available today. Therefore, the goal of this study is not to create a futuristic promise beyond today’s technological constraints, but rather to establish a path of convergence between human cognition and the modeling of machine thought.
Let’s begin modeling NeurrNet! Part 1 encompasses a design study of sorts, of our mathematical model and neural network, reinforcing fundamental concepts before delving into the detailed mathematics in the subsequent chapters.
Chapter 1: Hyper-dimensional Algebraic Networks for Cognitive Thought Representation
Let’s think of this step as akin to how a software engineer selects a data structure before delving into the actual problem-solving process.
So, we commence with the intention of delving deeper into the cognitive realm, only to find ourselves circling back to the question of how we can refine our thought modeling. We’ll revisit this question many times, as you may have noticed in the title. But how exactly do hyper-dimensional algebraic networks fit into the existing cognitive landscape?
The concept of hyper-dimensional algebraic networks, initially proposed in the seminal paper “Hypernetworks: A Molecular Evolutionary Architecture for Cognitive Learning and Memory” in 2008, aims to establish a sophisticated platform for the representation of cognitive thought that surpasses traditional neural network models.
A few key principles underpinning hyper-dimensional algebraic networks include:
1. Encoding: Mathematical transformations that map cognitive thought vectors into hyper-dimensional spaces.
2. Superposition: A process that enables the combination of multiple cognitive thought patterns to form composite thought vectors.
3. Retrieval: Leveraging the associative memory matrices to retrieve contextually related cognitive thought patterns based on a given query Q.
That’s great! But what is it, that sets hyper-dimensional algebraic networks apart? Do they rely on the same binary representation we commonly encounter in modern-day computing?
Well, a journal publication by Pentti Kanerva, the originator of the sparse distributed memory model, in 2009 sheds light on this. It underscores a crucial aspect: the remarkable properties of hyper-dimensional representation can be elegantly demonstrated using 10,000-bit patterns, which translates to 10,000-dimensional binary vectors. This represents an extensive space comprising all possible 2¹⁰⁰⁰⁰ patterns, often referred to as points within the space. This constitutes an astonishingly vast number of potential patterns, far more than any conceivable system would ever require to represent meaningful entities.
Here is an adapted representation of the state-of-the-art hyper-dimensional framework for cognitive thought representation:
Preface:
Hyper-dimensional algebraic networks emphasize the role of learning
and adaptation in cognitive thought representation. The learning process is essential for refining thought patterns and memory associations. Let Ti denote a high-dimensional cognitive thought vector corresponding to the cognitive thought i within the hyper-dimensional cognitive thought landscape. We’ll also define E as the transformation matrix responsible for encoding the cognitive thought vector Ti into the hyper-dimensional space. Additionally, consider a set of cognitive thought clusters denoted as C = {C1, C2, …, Cn}, embedded within the hyper-dimensional cognitive landscape.
Inference:
Given these constraints, we can infer the following about the cognitive landscape.
- The hyper-dimensional encoding process can thus be mathematically defined by Equation 4, where Hi signifies the hyper-dimensional representation of the cognitive thought vector Ti within the hyper-network architecture.
- The cognitive transformation process for the cognitive thought vector Ti can be formulated as in Equation 5, where Yi represents the transformed cognitive thought pattern derived from the input cognitive thought vector Ti using the cognitive transformation operators Φ, a weight matrix W, and a bias term b.
- Let’s also introduce the concept of cognitive clustering within the hyper-dimensional cognitive landscape, denoted as a clustering operator, Θ, which facilitates the organization of cognitive thought vectors into distinct clusters based on shared cognitive attributes and semantic similarities. Mathematically, the cognitive clustering operation can be defined as in Equation 6, where Θ maps the cognitive thought vector Ti to the corresponding cognitive thought cluster Cj within the hyper-dimensional cognitive landscape. As we progress through this article, we will repeatedly encounter the notion of identifying parent clusters and sub-clustering within thought clusters.
While the math does seem daunting (trust me I know), it’s worth taking a moment to explore Φ in greater detail before moving forward. These cognitive transformation operators, represented by Φ, facilitate the dynamic manipulation and propagation of cognitive signals across the interconnected neural units within the hyper-dimensional algebraic network. They play a crucial role in shaping the cognitive thought patterns, enabling the network to process and adapt to various cognitive inputs and experiences.
However, the question of the tangible significance remains unanswered: where does the physical significance lie within this framework? Upon closer mathematical examination, it becomes evident that modeling NeurrNet resembles an ever-expanding tree of thoughts, within the cognitive space we’d go on to restrict to a single thought space. This appears to touch on the fundamental design choice I hinted at earlier. But does it truly address it?
Yes, and no. Attempting to model an unconstrained expanding tree would likely result in an unattainable outcome. What if we constrained our search for discrete thought outcomes? Furthermore, as we delve into mood modeling and seek to replicate the attributes of memory and forgetting, we’d impose three fundamental design restrictions to achieve a tangible result.
- First, we’d limit our study of the cognitive space to just one thought space. There’s a finite probability of a thought-line transitioning to another, potentially unrelated thought space (also understood by us previously as a cognitive cluster) and engaging in a new line of thought, leading to unforeseeable outcomes.
- Second, while pursuing discrete outcomes, the model’s incorporation of an emotion we’ll term “mood” hereon could potentially result in abrupt thought-line conclusions, yet yielding what we might term “artifacts” that could plausibly influence future thoughts with a finite probability. Rest assured, we won’t overlook the study of mood and the impact of thought artifacts. In fact, as we navigate the fundamental intricacies, mood might very well emerge as one of the most extensively explored subjects in our study, paying homage to the investigation of sentient networks.
- The third one is a conditional. We may or may not assume that Ti is solely influenced by another thought Tj originating from the same cognitive cluster, or by other cognitive clusters exhibiting a significant correlation to the cluster under observation. We have the flexibility to adopt a case-by-case approach, particularly in highly situational contexts involving subjective physical or environmental factors. In such cases, this rule may not necessarily hold. However, within the study of constrained environments where the mood and factors affecting the mood demonstrate “low entropy,” we would strictly adhere to this rule.
Mapping the Interrelation Between NeurrNet and Human Sentiment
Before we conclude Part 1 of “x” sections, it’s essential to thoroughly investigate the modeling of thought priors and establish realistic boundaries for their incorporation into our neural network. At times, the term “artificial intelligence” does seem like a misnomer that fails to capture the depth of our cognitive abilities relative to the current state of exploration.
In any case, we’ll focus on modeling five such contextual factors that could indeed be incorporated to create a dynamic neuro-cognitive model that mirrors how human experiences and contexts influence our thoughts over time, akin to human sentiment.
- Priors should consider the influence of time, something we’ll call the temporal context (t) of the thought prior. Memories and experiences gradually fade, or become more vivid over time. This can be modeled as an exponential decay function as in Equation 7, where P(Ti,t) represents the prior probability of thought Ti at time t, Po is the initial probability, and λ is the decay rate (you can observe the inspiration from the classical half-life equation).
- They must also consider the influence of emotions; we’ll refer to this as the emotional context (e). Emotions play a significant role in shaping our thought processes. It’s no surprise, therefore, that positive or negative emotional states can influence thought patterns. We can model this using a sigmoid function as shown in Equation 8, where P(Ti,e) represents the prior probability of thought Ti given the emotional context e, e0 is the emotional baseline, and k controls the sensitivity to emotions.
- The influence of external factors and environmental stimuli on our thoughts cannot be ignored. We can represent the environmental context (env) as a weighted sum, defined in Equation 9, where P(Ti,env) signifies the prior probability of thought Ti given the environmental context env, C(Ti,envj) represents the influence of environment envj on thought Ti, and wj denotes the weights assigned to different environmental factors.
- Interactions with others, social situations, and relationships can shape our thoughts. We can model social context (s) using a simple linear combination, defined in Equation 10, where P(Ti,s) represents the prior probability of thought Ti considering social context s, S(Ti,sk) represents the influence of social factor sk on thought Ti, and ak are coefficients.
- Last but surely not the least, the current focus of our attention can
heavily influence our thoughts. We can model this attention and focus context (f) by considering the intensity of attention, defined in Equation 11, where P(Ti,f) represents the prior probability of thought Ti given the level of attention f, f0 is the baseline attention level, and β controls the focus sensitivity.
Thus, as a result of this section, we’ve established the fundamental design structure, clarified the path to a constrained full-scale implementation, and laid the building blocks for the mathematical and neuro-cognitive model, that is NeurrNet.
In the next section, we will explore the following topics:
- Tensor Calculus in Hyper-network Architectures for Neural Cognitive Dynamics
- Multi-linear Learning Models for Hyper-dimensional Cognitive Processing
Stay tuned!
Questions
- Let’s keep in touch if you have any questions or suggestions on LinkedIn.
