Thinking Like AI: How Kahneman’s Theories Parallel Machine Learning

In response to a recent comment on my post about ChatGPT’s famous strawberry problem, someone questioned the value of AI if it can’t solve fundamental common-sense issues. My initial thought was, if you had asked Albert Einstein, who lived next door, would he have been able to answer?

Even common-sense problems can stump people, no matter how intelligent they are. If you understand the deep mathematics and logic behind large language models (LLMs), such questions might not arise.

However, with new advancements, like the latest release of ChatGPT, LLMs are evolving to incorporate common sense and human-like reasoning. This reminded me of Daniel Kahneman’s concepts in Thinking, Fast and Slow, where he discusses two systems of thought: System 1, which is fast and intuitive, and System 2, which is slower and more deliberate. LLMs are now starting to mirror these two processes.

Kahneman uses the famous bat-and-ball problem to illustrate how System 1 often leads us to make quick judgments based on intuition without activating System 2 for deeper analysis. Humans can be intellectually lazy, relying too heavily on quick judgments. Interestingly, LLMs have operated similarly until now, often making fast predictions without deep consideration.

It’s also fascinating how many terms Kahneman uses to describe human cognitive biases mirror the challenges faced by LLMs.

For instance, Kahneman discusses the “halo effect,” where our minds oversimplify and make errors in judgment due to a lack of information. He describes this as exaggerated emotional coherence, where positive feelings toward someone, like their approachability, lead us to form an overly optimistic opinion of them. This parallels what we call hallucination in LLMs — when the model confidently generates incorrect information because it “believes” it to be highly probable, even though it doesn’t align with more rational analysis.

Kahneman also delves into confirmation bias, the human tendency to accept information that aligns with our pre-existing beliefs. Interestingly, in AI, we also talk about “weights and biases,” terms that are used technically but echo this cognitive phenomenon. AI can exhibit bias if not adequately trained, just as humans can exhibit confirmation bias in their thinking.

Another striking parallel is priming, where exposure to certain stimuli influences our later decisions. Kahneman explains this with a word fragment like “SO_P.” If you’re primed with the word “EAT,” you’re likely to complete it as “SOUP,” whereas if you were primed with “CLEAN,” you might think “SOAP.” This exact mechanism is at play in LLMs, where the input or “prime” it receives influences the model’s responses.

Kahneman also discusses cognitive effort. When focused on complex tasks, our minds use more energy, which he calls cognitive strain. In contrast, when we’re not using much mental energy, we’re in a state of cognitive ease. This concept is reflected in the latest LLM advancements, which apply more cognitive “energy” through techniques like chain-of-thought reasoning and reinforcement learning to improve performance.

Despite these similarities, there are still areas where AI and human cognition diverge significantly. One example is heuristics—mental shortcuts that help us judge quickly in complex situations. Rooted in mathematical calculations, AI excels at tasks like managing probabilities and base-rate statistics. For instance, humans often struggle with base-rate neglect, which tends to overlook general statistical information in favour of specific details. AI, on the other hand, is designed to handle these nuances effortlessly.

However, AI still has limitations regarding memory. In extended contexts, such as when writing a book, LLMs can sometimes lose track of middle events, retaining only the earlier and more recent information. However, this is an area of ongoing improvement.

Finally, Kahneman highlights a critical difference between humans and AI: we are not robots and don’t always make rational decisions. AI operates based on calculations and algorithms, but human decision-making is often driven by gut feelings and emotions, a departure from the purely rational utility theory.

AI and human decision-making processes involve weights and biases in different ways. I'm curious if Kahneman is aware of how his theories align with the development of AI.

That said, until AI achieves full vision capabilities, we can’t expect it to solve everything. To truly harness AI's potential, we need to strengthen our understanding and reduce biases.

And most importantly, don’t judge a book by its cover!