Fixing the news by rating stories
How can we untangle quality content from a web of misinformation and clickbait? We talked to Frederic Filloux about the News Quality Scoring platform.
Throughout his John S. Knight journalism fellowship at Stanford University, Frederic Filloux, journalist, media thinker, GEN Board member, and editor of the Monday Note has been working on the News Quality Scoring (NQS) platform, whose aim is to identify journalistic quality online at scale and in real-time using machine learning algorithms.
News have become homogenised making brands suffer
According to Filloux, ‘Facebook doesn’t really care about news. What matters to them is sharing things between friends and family. The algorithm is tailored to that end. In this context, news is an annoyance to them: it’s by essence unstable, and prone to controversy. It’s a stone in their shoe.’
‘The combination of mobile and social have reduced news to snippets’, which has led to the homogenisation of formats, where investigative news reports, tabloid fodder, and clickbait alike look the same on the newsfeed.
The result of this is that people who get news from social media struggle to remember the name of the news organisation that has published it, showing that the ‘notions of brand and authorship have been completely diluted’.
Clickbait is economically worth as much as serious content
This has brought about an economic imbalance in the news industry, where clickbait and investigative journalism have become homogenised to the extent that they carry the same advertising price tag.
How can publishers therefore be encouraged to carry out valuable investigative work that requires several journalists, weeks of research, and an infographics or data team when you can make the same amount of ad money with a five-second video of kittens?
How can news brands create loyalty and stand out from others in an environment of indifference?
The fake news elephant in the room
The homogenisation of news has also caused the contours of fake news and real news to blur.
How will platforms be able to distinguish between fake news and quality content, especially when fake news is becoming more and more sophisticated?
According to Filloux, Facebook has so far failed to take fake news seriously enough: ‘it should have been fairly easy for them to spot from the outset that many articles bouncing around the platform were coming from extreme right, fake news providers.’
Fake news was ‘grossly done’ over the last year, but it is going through a transformation and about to get a lot smarter. Most fake news providers have serious resources — often state funded — with access to cutting edge technologies to enhance their efforts.
‘The future of tech is going to serve fake news providers.’
During a class, which Filloux attended at Stanford University about political campaigning in the age of the Internet, a speaker from Cambridge Analytica (who claim to have ‘thousands of data points on each American voter’), stated that the ‘next election campaigns will no longer be about targeting groups of people, but micro-targeting individuals’, or as experts would call it: ‘weaponised artificial intelligence propaganda’. Seeing as online ads are only visible to individual users for a short period of time, they can be as false as they want, facing no accountability.
This underlines the growing importance for platforms to monitor what is being shared on their site in real-time in an equally sophisticated way.
‘Armies of bots will distribute micro-targeted fake news or hyper partisan stories about why the democrats have failed and why the healthcare system has failed the woman living below the poverty line in rural Alabama with a disabled son’.
The solution: News Quality Scoring
‘There is no universal definition of quality journalism.’
According to Filloux, if you were to ask somebody at Google or Facebook what they perceive quality journalism to be, they would look towards statistics, favouring a technology-driven approach.
Seeing as Filloux is not an engineer, he has decided to ‘invert the hierarchy’ putting technology at the bottom, and taking a journalistic approach when it comes to defining quality instead: ‘What makes a story relevant, interesting, engaging, and unique?’
‘Quality is therefore defined as value-added journalism, which is the result of genuine journalistic work through investigation, innovative treatment, analysis or expertise.’
What NQS is working to address:
Once in operation (sometime in 2018) the News Quality Scoring platform (NQS) aims to fix the imbalance in the news economy and surface quality at scale and in real-time.
The project will give an article a score of 1 to 5 based on its journalistic quality. The publisher’s ad server can then detect the highest scoring articles, and assign higher-priced ads to them, substantially increasing the revenue for the most valuable part of the editorial operation.
A tool not only built for publishers
Who will benefit the most from NQS?
- Publishers will be adequately rewarded for producing high calibre journalism.
- Advertisers will also be able to leverage NQS to increase their revenue, by raising the price of ads displayed next to content deemed qualitative.
- Readers will have better access to compelling and intelligent news stories, as well as being able to identify misinformation more easily.
- Brands will be able to guarantee that their content does not appear next to fake news stories, which could compromise their credibility.
How do you quantify value-added journalism?
Filloux came up with two sets of ‘signals’, which are either collected automatically or manually.
Automatic signals:
- Content page analysis: Author quality Score, publication quality Score, word count, video, data-rich story.
- Text analysis: Date line, time stamp, named entities, quotes.
- Propagation analysis: Engagement, discoverability, social propagation.
The subjective signals, such as writing style, emotion, and fairness are collected manually by editors.
Where does Artificial Intelligence come in?
All signals must be properly combined and weighted in order to create a reliable result.
For example, in a short article written by reputable authors, the wordcount signal will not carry as much importance, and will be partially cancelled out by a high authorship signal. This automatic adjustment of signals cannot be done manually making the project perfect for a machine learning application.
‘The machine learning aspect also ensures that NQS doesn’t only benefit big news organisations with established authors and recognised brands, but also newcomers.’
Filloux gave an example of an ex-engineer turned blogger writing about glass in architecture, which is an extremely narrow and niche domain. If the piece is well written, structured, and has a novel approach, the system should in an ideal world, give it the best possible rank. The Publication quality and authorship signal would carry a lower weight, and signals about the density of information would all score highly and carry more weight.
In order to provide more reliable information about authorship, Filloux wanted to originally build a ‘whitelist’ of journalists and bloggers, as he believes that bloggers in particular can make a big change in the world of news through providing more ambitious and niche content.
His aim was to create a database of the social footprint of each writer: are other journalists and experts following them? What does their LinkedIn resume say about them: where have they worked and have they been awarded any prizes?
Unfortunately no official body awarding prizes was able to provide him with a list of laureates and nominees of the last ten years making it incredibly difficult for him or anyone else to keep track of serious and ambitious journalists.
The automated weighting system is all the more important to provide accurate scores, seeing as there is not enough information about authorship to create a solid ‘whitelist of journalists’.
Keeping readers on site is a key objective for NQS
Filloux says that recommended content or curated reading is an excellent way for a publication to convey trustworthiness and create engagement by making readers stay on a page for longer.
In most news organisations, the selection of recommended articles (‘related stories’) is often done manually with a very limited filtering process, leaving the outlet with questionable or poor quality links to articles. Netflix, on the other hand, is a service that successfully targets viewers with related content, leading them to watch more and therefore ensuring greater brand loyalty. ‘However, the Netflix recommendation engine is manned by 300 people and costs $300 million a year to operate’, says Filloux.
NQS can help publishers build bundles of premium content; including related pieces, elements form the current news cycle, and pieces lifted from a publication’s archives based on quality scoring, which can be used as high quality recommended content.
Food for thought — there’s a lingering news problem NQS can’t fix.
NQS aims to ‘improve the news system’, which has been compromised by the emergence of super platforms, the expectation that information should be free, and a drastic change in reading habits.
However, not all problems have been brought about by the Internet:
Filloux believes that ‘the elitist bubble has killed the ability of journalists to look at their environment in a reliable way.’ There’s a disconnect between the social structure of the newsroom and the readership.
The riots in the outskirts of Paris in 2004 paint a clear picture of this problem: Le Monde widely publicised the fact that they had fixers in place to help its own reporters to cover the events happening outside the periphery of Paris, the same way they would be used to report in Kabul and Baghdad. This shows that in some instances, journalists in established newsrooms are so cut off from the rest of the population that they are unable to report on the true state of their own country.
