Generative AI and The Problem of Consciousness
One of the more interesting and vexing problems in both science and philosophy is the matter of consciousness. In basic terms, how does an intangible phenomenon like a mind (human or otherwise) arise, if at all, from a brain or body? How do we account for the subjective experience of the external world and the sense of self that we each possess? How does the question of consciousness bear, if at all, on the possibility of artificial general intelligence (AGI)? These and other questions are bound up in a recent, very public dispute between leading proponents of “computational functionalism” and those supporting Integrated Information Theory (IIT), with the former group accusing IIT of being “pseudoscience.” Theories of consciousness bear significantly on the topic of responsible AI, and particularly on “alignment” for AGI and superintelligent AI. The debate, therefore, has practical ramifications and is not just of academic interest. The complex topic of consciousness — on its own as well as applied to AI — is a difficult one to address in brief. In this piece, I use the current conflict over IIT to flesh out some of the important competing ideas in the field of consciousness as a whole.
The “hard problem of consciousness” (as it has come to be known) is the difficulty faced by philosophers, neuroscientists, psychologists, medical professionals and computer scientists in explaining the physical mechanisms by which the mind and consciousness exist, either in tandem with or apart from the brain or body. Neuroscientists and cognitive scientists in particular have gotten better at understanding and explaining the structures, functions and dynamics of the brain (as a biological entity) and cognition (how we process thoughts, sensations, and experiences). But the “whole-is-greater-than-the-sum-of-its-parts” aspects of the mind, even in lower-level creatures, has not been fully accounted for, mechanistically speaking. As reflected in the outcome after 25 years of a quasi-famous bet between a prominent neuroscientist and a prominent philosopher, we still do not understand the neurological mechanisms underlying consciousness. The mind/body problem still persists.
Predictions concerning the emergence of AGI and AI superintelligence don’t all see “machine consciousness” as a prerequisite. Perhaps computing systems can simulate the human mind to a high degree of fidelity, as they already do for many discrete cognitive capabilities such as playing Go and driving. But many consider human consciousness a key feature of the human mind and human intelligence. For a truly intelligent AI to meet or exceed human cognitive capabilities, the thinking goes, an AI must exhibit some form of what we might consider sentience or consciousness. As a shorthand, typical notions of what consciousness means for a given system include self-awareness, awareness of states in the external world, and an ability to process information obtained through interactions between the self and the external world. The big problem here, as noted, is that we do not yet have a definitive understanding of what constitutes consciousness in human beings, how it arises, or its physical, functional relationship with the brain and body. Without understanding the physical basis of consciousness for humans or other creatures, how is it possible to design consciousness in a computing system or to understand whether or not it has emerged?
A number of theories of consciousness have been developed by practitioners in philosophy, cognitive science, neuroscience, psychology, computer science and social science. Some have ebbed and flowed in popularity and influence. None have been definitely proven or disproven by empirical evidence. As outlined below, the emerging fractiousness and posturing about the science of consciousness is very likely owing to the recent, exciting developments in Generative AI. In particular, there is great anticipation that AGI or superintelligent AI may be within reach. Those in fields concerned with consciousness generally or machine consciousness more particularly are eager to offer a definitive explanation of how we will know when AI has achieved consciousness and perhaps what design paths to take to accelerate such achievement.
Computing Systems & Theories of Consciousness
From the time in the early 19th Century that Charles Babbage, the “father of computers,” designed a mechanical general purpose computer that he called the Analytical Engine, humans have mused about the possibilities of computer intelligence. The Analytical Engine, as designed, had an arithmetic logic unit, control flow in the form of conditional branching and loops, and integrated memory. It was the first general purpose computer that, in hindsight, was “Turing-complete.” Ada Lovelace, a talented mathematician who befriended Babbage and developed algorithms for running the Analytical engine, formed a vision of computers that went beyond number crunching. What were the possibilities of machines that could capably perform fairly sophisticated processes normally associated with human cognition? The design and uses of computation devices didn’t take great strides forward until the time period of World War II, when thinkers and engineers such as Alan Turing developed theories of computer science based in part on full working models of digital computers (many other individuals deserve credit, as is outlined here and here). It was at this time that the general study of artificial intelligence and the science of consciousness really took off as research disciplines.
Since the time of Alan Turing’s seminal research and writings in the theory of computer science, both the sophistication of computing devices and the capabilities of the hardware and software used to run them have advanced very rapidly. Along the way, theorists of both computer science and consciousness have gone from being fascinated by the possibilities of implementing sophisticated human cognitive capabilities in computing systems (such as IBM’s Big Blue that in 1997 defeated then world champion Gary Kasparov in chess) to thinking about the human mind as functionally identical to computing systems in their operation. The computational theory of mind (CTM) first developed in the 1960’s proposes that the mind is not simply analogous to a computer program, but that it is an information processing system that operates as do computer programs, only using the brain’s neural activity instead of electronic processors. A computational, algorithmic process can be carried out by either silicon chips or biological neural networks. As long as there are outputs produced by manipulation of inputs and internal states in a consistent, rule-driven manner, then mental functions can arise in computational systems just as they do in the human brain. Cognition and consciousness together are a form of computation. So goes the logic of CTM. Importantly CTM holds that computational emulations of human mental activity are sufficient to create the presence of a mind in a computing system. This important precept is at the root of growing anticipation that we will see consciousness emerge from increasingly more high-powered, capable AI systems. But how much confidence can or should we have in this precept, which forms the basis for most leading scientific theories of consciousness?
Since the 1960’s notions of CTM have become quite varied, with many nuances between different branches of thought. CTM has also been subject to widespread critique, but has nonetheless influenced the development of cognitive science and important movements in neuroscience, philosophy of mind, computer science, and psychology, among other disciplines. CTM is also known as “computational functionalism.” This brings us back to the IIT-related dispute.
Computational Functionalism vs. IIT: a battle for AI Hearts & Minds
Computational functionalism figures prominently in an ambitious article published in August 2023 by a group of leading neuroscientists, psychologists and computer science researchers. Entitled “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness,” the article claims to set forth key concepts that would serve as indicators of whether or not consciousness has been achieved in AI systems. These concepts are drawn from a number of leading theories of consciousness from both neuroscience and computer science (recurrent processing theory, global workspace theory, computational higher order theories, attention schema theory, predictive processing, agency and embodiments). The authors choose to bundle all these theories under the umbrella of computational functionalism given the common presumption that consciousness can arise in a constructed, inorganic system:
“An important upshot of computational functionalism, then, is that whether a system is conscious or not depends on features that are more abstract than the lowest-level details of its physical make-up. The material substrate of a system does not matter for consciousness except insofar as the substrate affects which algorithms the system can implement. This means that consciousness is, in principle, multiply realisable: it can exist in multiple substrates, not just in biological brains. * * * We tentatively assume that computers as we know them are in principle capable of implementing algorithms sufficient for consciousness, but we do not claim that this is certain.” (Article at p.13)
Below is the basic indicator property list and their connection to the various theories of consciousness from which each was drawn. (These specific details don’t matter for purposes of this piece, but those who may be interested in learning more about them would benefit from their elucidation in the Article, or from news summaries provided here and here.)
The article is both confident in many of its assertions about the relevance of these indicator properties and the theories from which they are drawn, but careful to qualify its reliance on computational functionalism:
“If computational functionalism is true, and if these theories are correct, these features would also be necessary or sufficient for consciousness in AI systems. Noncomputational differences between humans and AI systems would not matter. The assumption of computational functionalism, therefore, allows us to draw inferences from computational scientific theories to claims about the likely conditions for consciousness in AI. On the other hand, if computational functionalism is false, there is no guarantee that computational features which are correlated with consciousness in humans will be good indicators of consciousness in AI.” (Article, p.14)
As the authors themselves note, the cluster of theories from which the indicator properties derive are leading theories in which many have conviction, but may nonetheless rest on a false assumption. They explicitly omit integrated information theory (IIT) from their work “because it is not compatible with computational functionalism.” Per the authors, proponents of IIT hold “that a system that implemented the same algorithm as the human brain would not be conscious if its components were of the wrong kind” and “that digital computers are unlikely to be conscious, whatever programs they run.” (Article at p.33) With that, it is worthwhile providing more detail about IIT as a theory.
IIT and Panpsychism
The latest formulation of IIT (4.0) starts by asserting the reality of conscious experience (the “phenomenology” of consciousness) and works its way down from there to the physical underpinnings of conscious experience. The theory involves a complex series of nestled concepts ranging from axioms at the phenomenological level to mechanisms at the physical level. At each level, there are mathematical formulations that explain how distinct components combine to create a cohesive phenomenological experience. The variable Φ represents the level of information integration that has occurred from underlying physical mechanisms through higher level axioms for a particular system. The integrated information of a system is distinct from (greater than) the constituent information items of which it comprised. A system is meaningfully conscious only if it possesses a sufficiently large Φ. At this level of explanation, the key thing that distinguishes IIT is its starting point: instead of beginning with physical mechanisms (such as the brain or its neurons) and extrapolating from there, or from the middle of the brain/mind duality (e.g., examining distinct cognitive capabilities), IIT starts with consciousness and works its way down. IIT claims a mathematical foundation and offers means by which it can be empirically tested.
A long standing concern about IIT is its relationship to panpsychism, a broader philosophical theory of consciousness. Panpsychism posits that consciousness inheres in all matter and not just in living things. Rather, consciousness is part of the universe and elements of it are suffused throughout the natural world around us. Consciousness may not be sophisticated or recognizable to humans at the atomic level or in non-living matter, but basic levels of it are everywhere. Certain assemblies of matter (such as living creatures) can concentrate higher, more sophisticated forms of consciousness than, say, plants or rocks. But consciousness does not emanate or “emerge” from brains or bodies. Instead, brains and bodies are evolved to better coalesce and express the consciousness that inheres in the matter of which brains and bodies are made. Everything that exists has some form of internal experience or self-awareness, however basic or primitive. Such ideas go back many centuries. And they are not the product of fringe thinkers; many noteworthy philosophers throughout history going back to Plato and Aristotle have articulated versions of panpsychism. One of the biggest problems facing panpsychism is what is known as the “combination problem.” Essentially, it is difficult for panpsychism proponents to explain how consciousness at the atomic or cellular level combines to form the higher consciousness that we associate with the human mind. Still, panpsychism proponents contend, we have many counterintuitive theories (such as relativity, quantum entanglement, evolution) for which we lack much direct evidence or fully fleshed out mechanisms of action but which we still accept as settled science. There has been a recent revival of panpsychism, which many attribute to the publication of Philip Goff’s book “Galileo’s Error” in 2019 (author’s summary here; critical review of book here).
IIT does not overtly embrace panpsychism as such. What they share in common is the notion that consciousness may exist out in the world, in inert matter and plants as well as living creatures with brains. The physical assemblies of an inorganic system (electronic circuits) can potentially be conscious in the same way that an organic system (a creature with a neurological system) can be. And lower-level physical entities can possess consciousness. Both the notion that consciousness is highly distributed in the natural world (and not localized to creatures with higher brain functions), and that physical systems with lower levels of consciousness can combine (i.e., integrate) to support higher levels of consciousness are the attributes of IIT that liken it to panpsychism.
Nexus to the Rumble
As it turns out, six of the nineteen authors named on the Consciousness in AI article (Stephen Fleming, Mathias Michel, Chris Frith, Yoshua Bengio, Grace Lindsay, and Megan A. K. Peters) are either primary authors on or named endorsers of the letter published just a few weeks later denouncing IIT as pseudoscience. What is presented as methodological incompatibility in the article is then more severely critiqued as unworthy pseudoscience in the letter. The principal objections in the pseudoscience letter is that IIT is gaining too much prominence as a “leading theory” of consciousness and results from a recent adversarial study finding empirical evidence for some of IIT’s predictions are both flawed and misleading. The letter also paints IIT as “panpsychist” in its orientation. Panpsychism is identified in the article as a “metaphysical” theory of consciousness as opposed to a “scientific” one. As such, panpsychism’s association with IIT forms a critical basis for the latter’s characterization as pseudoscience. One author of the letter separately defends and articulates in greater depth the basis for the pseudoscience charge.
IIT certainly has its quirks and counterintuitive tenets. But, as the case is nicely laid out in a recent piece on the unfolding fracas, IIT has sufficient structure and detail as a theory so as to be empirically testable. As such, however skeptical one may be, it is somewhat extreme to deem it pseudoscience. This is particularly so given that no theory under the umbrella of computational functionalism has been conclusively proven empirically, and the major precept of computational functionalism — “substrate-independent” consciousness — is itself an unproven and disputed conjecture. Like computational functionalism, IIT embraces that consciousness may not be unique to living creatures with more sophisticated brains. Unlike computational functionalism, IIT embraces that consciousness already exists in inorganic matter, whereas the former holds that consciousness may emerge in an inorganic computing system, but only one that algorithmically replicates critical functions of the human mind. Additionally, unlike computational functionalism, IIT’s founder and main proponent does not believe that consciousness is substrate-independent, or, thereby, that computing systems can achieve what would be equivalent to human consciousness. Thus, IIT is more focused on questions of human consciousness and it is entirely unclear whether any theory associated with computational functionalism can help engineer or predict the emergence of consciousness. The unfolding fracas appears ultimately to concern steering the attention, perceptions of relevance, and resources of those developing powerful Generative AI foundation models and actively espousing the near term possibility of AGI.
In conclusion, we return to a question posed earlier: Without understanding the physical basis of consciousness for humans or other creatures, how is it possible to design consciousness in a computing system or to understand whether or not it has emerged? On this question, IIT theorists may very well be right; any semblance of consciousness to emerge in a computing system would just be a powerful simulation, without the computing system having an actual sense of itself or the external world in the way that humans and other creatures do. Computational functionalism may also in the long run prove correct, though its various theories are unlikely to make a clear prediction of the requirements for machine consciousness or offer reliable criteria for when we might know it has arrived (e.g., the Turing Test was long held to be suitable for determining the existence of AI, but we now have AI that can assuredly meet Turning Test criteria without being truly “artificially intelligent”). In the meantime, no particular theory of consciousness is definitive science and perhaps it is premature to declare any of them pseudoscience. Whether machine consciousness is viable is more likely to come about through design and engineering advancements in AI hardware, system architecture and programming models, which won’t rest on theory.
Copyright © 2023 Duane R. Valz. Published here under a Creative Commons Attribution-NonCommercial 4.0 International License
The author works in the field of machine learning/artificial intelligence. The views expressed herein are his own and do not reflect any positions or perspectives of current or former employers.
