Why AI means different, not better
When bloodletting becomes antibiotics, not more blood
THERE IS A debate in economic circles about why productivity growth has seemed to slow over the past decade or two. Is it mis-measurement, or is it structural? The answer is used not so much to explain the past but understand the future. It might help resolve the question: will artificial intelligence lead to a boom in output and productivity, or will it be just a marginal bump?
The debate bothers me because I look upon the recent studies with a dose of common sense and an eye to ground-truth. And I’m frustrated by what I see.
Robert Gordon of the University of Chicago suggests that if AI were so important we’d notice it in the stats by now. He advanced that point in 2012 — the year that the foundational achievements in deep learning were just being published. He argues that mis-measurement of IT isn’t decisive because we mis-measured lots of technologies in the past, notably candlelight and lightbulbs. And he teases technologists by explaining that in his strolls in downtown Chicago there’s nary a robot in sight.
Yet his criticisms insult his scholarship. Incandescent light mis-measurement should give us less confidence in his metrics, not more. Regarding AI and output, it’s as if one were to poo-poo aviation in 1903, the year the Wright brothers arrived in Kitty Hawk, North Carolina.
Sometimes the papers seem to measure the data that’s available rather than the underlying question itself —rather like the drunk who lost his keys up the road but looks for them under the lamppost where the light is better.
Take the Brookings paper “Does the United States have a productivity slowdown or a measurement problem?,” by David Byrne, John Fernald, and Marshall Reinsdorf in Spring 2016 (here). The authors’ answer that US productivity slowdown is structural and not a measurement problem.
However in reaching this conclusion, they used shaky data and methodological assumptions. The analysis of mis-measurement is based mainly on domestically-produced IT equipment and other capital goods — not software, where the mis-measurement might look even more pronounced because Internet-delivered products are harder to count and track improvements. As for the gains from free digital services, the authors dismiss them as “non-market, home production” used during “nonmarket time.” Hence, by not counting it, they find that “market sector TFP growth has slowed.” (Emphasis mine.)
But take a moment and think about how people actually live with these gadgets, and you can’t help but embrace a different conclusion. A woman uses her iPhone to snap pictures of her damaged car to email to an insurance company to file a claim: the cost is basically invisible. But just a decade ago it would have likely meant going to a shop (car and fuel, say $5), developing the images (photography, say $10) and mailing it (postage, say $5). That’s $20, without even considering the time spent. That “transaction cost” for a minor car insurance claim today costs her effectively nil. But her $20 contribution to GDP disappears — so productivity dips. And again, this doesn’t even consider the opportunity cost.
Yet the benefit isn’t counted — because it piggybacks atop the “non-market, home production” gadget used during “nonmarket time” according to the Brookings paper. Really?
Back to AI. I have the feeling like there’s a lacuna in the economic analyses because it is more straightforward to count incremental improvements rather than disruptive innovation: better gas milage rather than self-driving cars. Medicine went from bloodletting to antibiotics as a revolution not an evolution; as a paradigm shift in the real, Kuhn-ian sense. Healthcare didn’t improve because we could perform bloodletting better, but we discarded the practice for more effective techniques based on science and evidence.
James Bessen of Boston University has a great paper (here) that shows that of the 270 occupations in the 1950 US Census only one has been eliminated by automation: elevator operator. The rest? They’re still counted, even though the work itself is radically different.
And a tidy response to the measurement debate comes from the veteran tech investor and polymath Bill Janeway, who summarizes “the best and the rest” data (here): mediocre firms camouflage the productivity boom of outliers in the overall economic stats. (Listen to Bill and I discuss the issue on The Economist’s tech podcast here.)
Are our metrics on AI’s impact on the economy up the task? The National Academy of Sciences released a report in April 2017 (here) that recommended creating an “AI Index” to track the technology’s progress akin to the way the Consumer Price Index tracks inflation. It’s a great idea, though just as prone to statistical wrinkles as CPI. Yet it’s worth trying.
What should be clear, however, is that AI is poised to introduce science and evidence to all areas of life, not just improving current practice but changing it. It doesn’t seem like our tools are capable of measuring this, nor that the discipline of economics has the self-confidence to admit as much.
Disagree? Concur? Keen for more? Contact me via Twitter: @kncukier
