News post-Facebook, news apps and research on quality from Korea
Looking at research from Dr. Sujin Choi of Kyung Hee University in Seoul
The big news in the news world this week was the deals between publishers and Google in Australia, as well as the lack of such deals with Facebook that led the platform to stop news content Down Under. There are plenty of opinion and pieces out there about the blackout and current situation (Facebook is bringing back the news), though to me the company’s moves in Australia are an interesting jumping off point to discuss alternatives to getting your news on social media.
While the Big Tech companies get a huge amount of attention for their impact on democracy and discourse, there are plenty of other efforts to get people news, from individual publishers to newer social platforms to Google to news apps. These all may get increased attention if the link between Facebook and news continues to break down and we move into a “post-Facebook” news environment.
To me the biggest question about how we get news over the next 10 years is how these various options differentiate themselves in what they prioritize showing to readers, and if they will differ from Facebook’s model of promoting content largely on what makes people click. All the competing ways to get news will jostle to create the “news experience” that is best for their users.
Part of that competition will be with algorithms that categorize the news, not just based on clicks but on different criteria that improve the experience. That is of course what Deepnews is doing with quality, though an interesting research paper from Dr. Sujin Choi of Kyung Hee University in Seoul also looks at the ideas of quality and “journalistic values” in an algorithmic way.
I spoke to Choi by video to understand exactly where her research was similar and where it was different from Deepnews. One difference is the context of the news environment in South Korea. While I often think of news in North America and Europe as being read through individual publishers’ sites and social media, in Korea (as in many other Asian countries) news aggregators play a huge role. In Korea the big name is Naver, which is is believed to strongly incorporate audience feedback (clicks) into what news is shown.
“The news audiences’ choices are influential in the context of South Korea and can affect which issues get widely distributed and receive attention. Also their choices can affect journalistic production and editorial news selection,” Choi said.
For that reason, while Deepnews created part of its training set using journalism school students to assess articles based on the “journalism school” definition of quality, Choi focused on members of the public to see whether they may hold the same ideas. They read 1,500 articles from Naver.com and rated them both for quality (1–10) and for “journalistic values” such as believability, readability, depth and sensationalism. Choi wondered if you can use machine learning to predict the quality of an article that the model hadn’t been trained on given the journalistic values that it shows. It turns out you can, and that the abstract ideas of “values “ are better predictors than “formal features” of articles such as the number of named entities it has or the length.
“We found that the stronger predictor of audience-rated news quality is journalistic values. Among them, believability, depth, and diversity were regarded more important than others. It suggests that news audiences, like journalists and journalism scholars, regard journalistic values highly as substantial factors in news quality,” she said.
The value of “sensationalism” had the lowest predictiveness.
This research is of course a little different from what Deepnews has done, which is to use a larger collection of more than 50,000 articles, labelled with a quality score to predict a quality score for other articles, rather than use one set of scores to predict another. However, it is further proof that there is a concept of journalistic quality that people understand and that it can be measured algorithmically.
Choi’s research also means something for the question at the beginning of this post, about how we get our news. If quality can be grasped algorithmically and people can understand it, then perhaps we should start expecting (or demanding) that news platforms incorporate the idea into their systems in addition to engagement.
“I think the primary logic of how the digital news platforms are operating should be revised by considering the quality of the news articles, considering democratic public discussions, the quality of public discussions,” Choi said. “I think it’s important to consider the news quality in their algorithm design.”
In the future different news sites and aggregators may be using a several algorithms to categorize different news articles according to different criteria such as quality but also others like those journalistic values, how scientific an article is or something else. Twitter CEO Jack Dorsey has fantasized about an “app store for social media algorithms” that would give people the choice over what kind of things they are being shown, rather than being pushed articles selected for them by a centralized algorithm that benefits one company.
Ten years from now, we may look back on using primarily clicks to categorize news online like someone trying to measure air quality using a thermometer.
P.S: If you’re interested in a piece with background on the situation in Australia, our very first blog post for Deepnews (September last year) spoke with Prof. Rob Nicholls of University of New South Wales on all things antitrust.
