Emergence of understanding in GPT-3: Wishful thinking or functional or ‘real’ and how? Centers of understanding.
There’s lots of debate in AI circles presently — in Q4 2022 and all year in fact— about large language models (LLMs) like GPT-3 and whether they really, genuinely ‘understand’ or not and whether mystical (or otherwise) ‘emergence’ is occurring or not.
In my first article on GPT-3 I submitted evidence that GPT-3 is strongly reliable (like > 95% accuracy) at novel examples of classically difficult common sense — or abductive — reasoning tasks that involve generating probabilistically likely explanations of situations.
Like:
Why did the man return to the car for a heavy wrench upon discovering the front door of his home ajar?”
Of course we know it’s likely because he feared an intruder and went back for a makeshift defensive weapon.
It turns out GPT-3 knows that too (in perfect English, it’s not just doing multiple choice like many other — poorly — competing systems):
But how does GPT-3 do it?
How does understanding emerge in an LLM?
And what is emergence anyway?
