Member-only story
When LLMs Run Out of Memory: Unpacking the 3.6-Bit-Per-Parameter Ceiling
Imagine your language model as a bucket that you can use to store all sorts of information by pouring in from news articles to obscure code snippets. But eventually, there’s only so much it can hold before it starts spilling out exact phrases, memories of the training set. How much can it really store? According to Morris and co-authors, there’s a cap of around 3.6 bits per parameter after which the model’s behavior shifts in interesting ways.
What the Study Does
When we say that a language model “remembers” something, what do we really mean? In the paper “How much do language models memorize?” (arXiv: 2505.24832), by Morris et al., May 2025, the authors argue that this term has been used loosely (like many other words used in the AI hype) and that we need sharper definitions to make sense of memorization.
Unintended memorization
This is the rawest form of memory: the model has essentially stored specific fragments of the training set and can reproduce them verbatim. Think of it as when a student crams for an exam and recalls sentences word-for-word without necessarily understanding them.
For LLMs, unintended memorization shows up when the model outputs unique strings (like an API key, a poem…
