A Simple Take on Measuring AI Progress

The way futurists tell it, AGI (Artificial General Intelligence) is either around the corner or never going to happen. Part of the problem with predicting AI progress is that AI is hard to quantify. It’s not like CPU progress, which we can approximate using Moore’s Law. Ray Kurzweil attempted to use Moore’s Law, though, to extrapolate AI gains and predict we’d have approximated human brains by 2020. But that extrapolation assumes that having the processing power required to perform a complex task is all that’s needed to perform a complex task. That’s just not true. AI progress requires both software and hardware progress.

Since we can’t measure AI in terms of gigahertz, it might make sense to take a hybrid approach. Perhaps we can measure AI progress in terms of cognitive accomplishment, as well as the size of that accomplishment’s associated hardware. For example, the size of Deep Blue, which is the computer that beat the world chess champion Kasparov in 1997, was about the size of a filing cabinet. Meanwhile, the chess-playing module inside of our brain is maybe the size of a pea. The volume ratio between a pea and a filing cabinet is perhaps 1:50,000, which means that the AI for playing chess had roughly one-fifty-thousandth the efficiency of the human brain. However, within a decade, cell phones had enough processing power to play world-class chess.