Member-only story

LLMs Can’t Learn Maths & Reasoning, Finally Proved!

Vishal Rajput
AIGuys
Published in
11 min readDec 16, 2024

--

The age-old question regarding LLMs: Do large language models (LLMs) solve reasoning tasks by learning robust generalizable algorithms, or do they memorize training data?

To investigate this question, recently a paper used arithmetic reasoning as a representative task. Using causal analysis, they identified a subset of the model (a circuit) that explains most of the model’s behavior for basic arithmetic logic and examined its functionality. Now we finally have the answer to how LLMs solve maths and reasoning tasks.

So, without further ado, let’s break down these new papers on Algorithmic reasoning.

Table Of Contents

  • Defining Reasoning
  • Types Of Reasoning
  • Understanding Heuristics
  • Breaking Down Black-Box AI Internals
  • Mathematical Circuits
  • Understanding Circuits In More Details
  • Conclusion

Defining Reasoning

In his 2019 paper On the Measure of Intelligence,” François Chollet defines intelligence as “skill-acquisition efficiency,” emphasizing the importance of generalization and adaptability over mere task-specific performance.

--

--

AIGuys
AIGuys

Published in AIGuys

Deflating the AI hype and bringing real research and insights on the latest SOTA AI research papers. We at AIGuys believe in quality over quantity and are always looking to create more nuanced and detail oriented content.

Responses (16)