The Hard Argument Against LLMs Being AGI

For some reason, the world seems to have gone crazy after LLMs reached the consumer market chasm. Generated at ImgFlip.

A few weeks ago, Microsoft published this, perhaps slightly extravagant, paper “Sparks of Artificial General AI.” Huge title, but when you read the paper, especially the metrics section, you notice that they just used an IQ-test-like system from 1994 to define “Hey, this is AGI!” which completely ignores the state-of-the-art AGI discussion in related scientific disciplines:

“In this paper, we have used the 1994 definition of intelligence by a group of psychologists [Got97] as a guiding framework to explore GPT-4’s artificial intelligence. This definition captures some important aspects of intelligence… but it is also vague and incomplete. … we acknowledge that this definition is not the final word on intelligence, but rather a useful starting point for our investigation.…

For instance, Legg and Hutter [Leg08] propose a goal-oriented definition of artificial general intelligence: Intelligence measures an agent’s ability to achieve goals in a wide range of environments. However, this definition does not necessarily capture the full spectrum of intelligence, as it excludes passive or reactive systems that can perform complex tasks or answer questions without any intrinsic motivation or goal. One could imagine as an artificial general intelligence, a brilliant oracle…

Moreover, the definition around achieving goals in a wide range of environments also implies a certain degree of universality or optimality, which may not be realistic… The need to recognize the importance of priors … was emphasized in the definition put forward by Chollet in [Cho19] which centers intelligence around skill-acquisition efficiency, or in other words puts the emphasis on a single component of the 1994 definition: learning from experience (which also happens to be one of the key weaknesses of LLMs).

Another candidate definition of artificial general intelligence from Legg and Hutter [LH07] is: a system that can do anything a human can do. However, this definition is also problematic, as it assumes that there is a single standard or measure of human intelligence or ability, … furthermore, this definition also implies a certain anthropocentric bias… While we do not adopt any of those definitions in the paper, we recognize that they provide important angles on intelligence… Equipping LLMs with agency and intrinsic motivation is a fascinating and important direction for future work.”

So they reject many definitions because those definitions do not fit LLMs — as if they had already decided that LLMs are AGI. I don’t understand why they need to call it AGI if it doesn’t fit the definition of contemporary science? AGI is a well-established concept in science, and now millions of humans globally are confused about what is happening. When Google, Meta, or IBM exceed average human performance in similar situations, for some reason, they do not call it AGI. Microsoft seems to be trying to reinvigorate interest in Bing, which has been failing for many years.

These kinds of publications have created some echo chambers around Twitter. Some vocal doomsday prophets say that LLMs will cause human extinction, and some fans suggest “LLMs will soon access the nuclear weapons API from the internet” and cause a Skynet scenario. There is a reason why we do not put nukes on the internet; hackers and hostile governments already exist.

Elon Musk signed a letter of concern last week based on this kind of worry. It got a response from the Quebec AI center, which encouraged AI companies to collaborate closer with the existing science. Last Friday, Elon walked the walk and opened Twitter recommendation systems for open review at GitHub. At the same time, he cashed out with the hype and founded his own AI startup with a good valuation. To me, all this seems to have more to do with brand building, social media marketing, and stock market valuations than real science.

Which side does OpenAI and Microsoft seem to be on with their extravagance? Original image; see the picture. Link to text.

The tactics of Microsoft and OpenAI resemble the ones used by climate denialists, who refuse scientific collaboration with IPCC-style organizations and prefer shouting out from a distance by using unequivocal scientific conceptual frameworks (which are sometimes correct from classic physics sense but do not pass as modern dynamic climate models).

Science deserves to be critiqued; that is the essential beauty of it. However, when you use words and replace their established scientific meaning with your own definitions, you ought to have better reasons than “LLMs are not AGI by these definitions they use, but we want to use that word in our marketing efforts!”

However, the question about the differences between LLMs and AGI is interesting anyway, and it always helps to learn from each other. In this essay series, I will try to show explicitly how LLMs differ from AGI systems and why transistor computers will never be able to reach that.

This way, I will give a hard list of assumptions to be falsified by those who claim that AGI is possible with transistor computers or LLMs (or neural networks trained on classic hardware). However, I am not arguing that we should not be worried about the ethical consequences of technological innovation (capitalism and automation have some existential problems for democracies through hyperreal alienation and similar issues).

I claim that we ought to be intellectually honest and conceptually coherent in our claims and focus our energy accordingly. My foundational thesis is the following:

1. Everything running on transistor computers is still limited by the conceptual framework of Turing Machines.
2. Large Language Models only work in the domain of syntactically coherent language manipulation (thus distributional semantics), while the semantic coherence of narratives (semantic composition) is beyond Turing Machines and thus also their scope of understanding. Semantic composition or a similar system is necessary but not a sufficient requirement for AGI.
3. Due to the nature of Few-Shot Learning, many people make the mistake of mixing up the knowledge of the observer with the knowledge of the test apparatus. This is also known as Prompt Leading in the case of LLMs and in Quantum Mechanics, known as Observer Effect. This is why many people mistake LLMs for having consciousness-like capabilities: the eventual semantic coherence comes from the human operating the tool, not the tool itself.
4. In quantum computers, theoretical computational grammars (for example, Categorical Grammar) exist to bridge the gap between natural language understanding and semantic composition. But they might be limited to understanding sequences of separated sentences and unable to understand sequences of paragraphs in the same sense as humans do. In other words, they might be unable to do discursive evaluations, which might be necessary for AGI.
5. Early days of Post-Cognitivism and especially Connectionism (the philosophical beginning of current neural networks and machine learning paradigm, which challenged the Cognitivistic symbolic AI) believed that scaling up the number of neurons would lead to the emergence of general problem-solving and human-like consciousness. Today we refer to these kinds of pure bottom-up emergence systems assuming specific structures as “magic dust” emergence.
6. “Non-magic dust” emergence works through the principle that specific evolutionary elements will create specific cognitive functions for existential beings. Freedom of human beings in society (as opposed to say, animals) is based on sharing these cognitive functions, which enable us to cooperate in society. Contemporary Post-Cognitivism and philosophy of mind take the complexity of our problem-solving skills and their epistemological access more seriously (as opposed to early Post-Cognitivism and Connectionism) because neuroscience and evolutionary history of brains have matured as science. For that reason, we do not believe LLMs are a route to AGI that would be able to generalize in the context of human-specific cognitive functions. They can only extend human capabilities.
7. Human ability to generalize knowledge from explicit phenomenological experiences to new contexts is related to the implicit noumenological subconsciousness. When we come up to a new situation, the posteriori parameters of the scene can recall correct noumenological structures, which are not inherently present in phenomenological natural language. They are rather only semantically composable from the whole narrative by an existential agent (who has gathered individualistic but societally functional posteriori from real life). According to post-Cognitvists, when the implicit noumenological subconsciousness of an agent is misaligned from the environment (of other agents), it will experience stress. It may end up in an existential crisis (which can surface as mental health symptoms).

In other words, I see through these theses the basic elements of AGI would demand existential noumenological interpretations of phenomenological narratives. When we encode a memory, both the phenomenological symbol and the noumenological sense experience present at the moment will be encoded to our brain in some currently unknown manner. When a machine wakes up to a signal, it will only analyze the phenomenological input patterns and reproduce the phenomenological output patterns. They are completely deterministic as computational logic systems.

Human beings have unstable emotional subconsciousness as a source of noumenological amplification of our ability to see and interpret phenomenological patterns. We are encouraged to try some phenomenological solution articulation in our conscious mind first by simulating it and then enacting it. Then we may repeat the process because the failure has shifted our noumenological stance, giving us a new simulation to be enacted. Understanding the finer details of this aspect is hard, but hopefully, after reading the essay, you will see the core idea clearly.

I will go through all this in the following sections of this essay series:

1. Limits of Turing Machines and Explaining Transformers via Dynamic Programming — it is easier to understand what LLM is and what they can not do when we know some basics of computer science as a discipline. We can safely assume that LLMs are unable to escape limits of Turing Machines.
2. Understanding LLMs and Few-Shot Learning — by using the concept of Algorithmic Fingerprints (from Dynamic Programming), it actually becomes easier to see that nothing in the model changes as the prompt grows, but instead, the user may suffer from Observation Effect called Prompt Leading.
3. Current models of semantic coherence — computer science is a good tool for proving what is impossible with some systems. While transistor computers can do distributional semantics, they cannot do semantic composition efficiently (which belongs to the domain of quantum computers). Sure, scientists have been wrong before, but we need evidence that scientists have been wrong before drawing that conclusion.
4. From Connectionism to Existential Cognitivism — I finally explain why humans and AGI are significantly different than LLMs (or any neural networks, for that matter) and what the contemporary science, which Microsoft is wildly skipping here, says about this whole topic.
5. I might write a bonus sector later: Auto-GPT is taking the whole thing in the correct direction for AGI, but it fails because it uses LLMs as a top-down discriminator of poor bottom-up solution candidates. LLMs are fundamentally bottom-up utterance generators.