Algorithms vs Editors

New research examines the power — and limits — of AI on news story recommendations

By Timothy Aeppel

Algorithms can be better than human editors at predicting what news stories people will read online — but the advantage has distinct limits and can lead readers to consume a less diverse mix of news.

Those are two key findings of new research by a team studying how use of artificial intelligence (AI) impacts consumption of online news.

The main takeaway from the work is that humans and algorithms can complement one another, said Ananya Sen, a postdoctoral student at MIT’s Initiative on the Digital Economy, who conducted the study with Joerg Claussen of the Munich School of Management, and Christian Peukert of the Catolica-Lisbon School of Business and Economics. Human editors, for instance, consistently do a better job than algorithms when there’s an unexpected breaking news story, since there’s limited data on breaking news that can help guide the machines.

Strategic Advantage?

“Data can give [a news organization] some level of strategic advantage,” said Sen, “but it doesn’t seem to be a runaway hit that will give you dominance in the market.”

In a new paper, The Editor vs. the Algorithm: Targeting, Data and Externalities in Online News, published in June, the researchers laid out how they tested their ideas at an unnamed German news organization which receives more than 20 million unique visitors to its website each month. “It is important to note that it is rare for major legacy news outlets in the world to experiment with algorithmic curation of their homepage,” the researchers wrote in the paper. It is a new field of research.

The researchers separated a control group of readers — which saw the organization’s website as assembled by human editors — and a “treatment group,” which saw a version in which one of the four slots on the page was personalized using an algorithm that studied the reader’s preferences.