The Technology Behind OpenAI’s Fiction-Writing, Fake-News-Spewing A.I., Explained
The language model can write like a human, but it doesn’t have a clue what it’s saying
By Karen Hao
On Feb. 14, the nonprofit research firm OpenAI released a new language model capable of generating convincing passages of prose. So convincing, in fact, that the researchers have refrained from open-sourcing the code, in hopes of stalling its potential weaponization as a means of mass-producing fake news.
While the impressive results are a remarkable leap beyond what existing language models have achieved, the technique involved isn’t exactly new. Instead, the breakthrough was driven primarily by feeding the algorithm ever more training data — a trick that has also been responsible for most of the other recent advancements in teaching A.I. to read and write. “It’s kind of surprising people in terms of what you can do with […] more data and bigger models,” says Percy Liang, a computer science professor at Stanford.
