It’s Expensive to Be Poor. A Business Case for AI in the Newsroom

Today’s news consumers are exposed to countless, disconnected sources of news. And as if the online news picture weren’t fragmented enough, 2016 happened and threw us face-first into a ‘post-truth’ era. In the wake of fake news scandals, citizens — increasingly wary of their own echo-chambers, expect trusted media brands not only to report accurately, but to help audiences digest the overwhelming news picture.

Traditional news publishers recognise the importance of taking on a stronger curator role for their audiences. However, analysing and decoding the fragmented news picture places an additional resource strain on already slimmed-down newsrooms. To keep up with news consumers’ needs, news publishers need to do more, and better, with less.

Enter artificial intelligence, the all-hailed antidote to resource dilemmas across industries. Ask any chief digital officer (or worse, CFO) and sooner rather than later, automation, algorithms, and the proverbial job-stealing robots will pop into the conversation as the hammer to all nails in the coffin of inefficiencies and repetitive, boring jobs.

A publisher’s dilemma

Machines and automation undoubtedly provide a possible answer to journalism’s resource challenge, as they have done for other industries. Emerging reports about automation in the newsroom show how news publishers are starting to experiment with AI applications like machine learning, natural language processing and generation, speech-to-text and text-to-speech, image recognition and robotics.

Despite growing enthusiasm for automation in the newsroom, publishers’ AI initiatives often stop at the prototype level. According to a recent O’Reilly study, online media accounts for only 1,33% of AI investments, while platforms like Facebook, Google and Amazon lead the pack by a whopping 32%.

AI requires heavy investments in highly sought-after machine learning specialists, with the time and competence to understand the news industry’s problems and train machine learning models to solve them. Creating reliable training data sets is no easy feat. Re-hashing journalists’ workflow to make room for automation, requires tech savviness among editorial and management teams, complete with a good dose of patience and the emotional intelligence required by structural changes. None of this comes cheap.

All the President’s Men

AI in the newsroom has the making of a great story, but following the money trail brings up a very mundane reason for low AI investments: many news publishers view AI as a risky luxury they can’t afford.

So how can news publishers beat the money catch-22 and find the means to invest in AI applications that can ease their resource strain?

One option is to channel more of the readers’ newfound willingness to pay for good journalism into AI investments. After the 2016 US presidential election, traditional media brands saw a surge in subscriptions, which boosted revenues in the first quarter of 2017. While the Trump Bump and related crowdfunding and philanthropic initiatives help, these alone cannot account for profitable newsroom operations.

Source: Nieman Lab

The other option — and the more sustainable answer to news publishers’ resource conundrum, is to choose wisely. In investment terms, this means to view the AI-powered newsroom as a portfolio of AI use cases, and commit more resources and longer thoughts into fewer AI applications with a strong business case.

The business case for AI in the newsroom

Building a business case for the AI-powered newsroom is largely the same exercise as comparing the costs and benefits of automating any other business process. As a minimum, this exercise involves answering three key questions:

1. How will AI impact the news value chain?

Regardless of news medium or editorial workflow, the journalism value chain boils down to three steps: Research — Produce — Distribute.

For the past few years, we have seen newsrooms focus mostly on using machine learning in digital distribution i.e. using algorithms to present personalised reading recommendations or targeted ads. This is not surprising, given traditional news media’s complicated relationship status with Facebook. As they try to stay on top of the ever-growing number of distribution platforms, news publishers have taken a copycat approach to the personalisation of editorial and commercial content. Without the resources to explore and develop AI applications of their own, publishers have jumped on the personalisation wave legitimised by social media platforms and their loyal following of digital marketers.

There is nothing wrong with using predictive algorithms for growth hacking, but publishers should be critical to applying marketing techniques to editorial content. Personalised communication may work for push notifications and special interest editorial content like weather, sports and entertainment, but personalising general interest news can implode in filter bubbles.

As technologies for natural language processing and generation mature, and skepticism towards robot journalism decreases, we will likely see AI moving upstream in the news value chain. There is a strong case for letting machines do the tedious research and get better at fact-checking, as well as automated news writing based on structured data sets.

2. How will AI make or save money?

The obvious economic advantage of automation is cost reduction by time saved on tasks that machines do better and faster than humans. The research of 11.5 million documents by 400 journalists in the Panama Papers investigation was predominantly a manual process. Sifting through this magnitude of data begs for developing natural language processing models for content analysis and classification. AI applications like image recognition or natural language processing have the potential to do years of research work in seconds and thus have a direct positive effect on operating costs.

Not only does automation in the newsroom save precious time journalists can spend on interviews and storytelling instead, but it can also have a direct positive impact on the top line. For example, text-to-speech opens up for new audio services news subscribers may be willing to pay for. Fully automating translation (estimated by McKinsey to happen by 2018) can both reduce the cost of translation services, but also open for new subscriber markets and revenues in international markets.

3. How will AI help build trust in the news?

For many industries, making more money from than spending on AI, is by definition a strong enough business case for automation. The business case for AI in the newsroom differs from other commercial industries, in that it operates with an additional currency — trust.

Trust has been the fundament of the news industry since long before the sharing economy coined it as the new currency of the digital age.

Trust builds loyalty, and loyalty pays. News consumers’ trust drives willingness to pay, which turns into subscription revenues, ‘eyeballs’ turn into advertising money, and readers’ goodwill turns into crowdfunding or philanthropy.

How to spend it. An AI investment framework for the newsroom

At digital product development studio Bakken & Bæck, we helped several Scandinavian newsrooms bring AI into their daily workflow by using Orbit, an AI platform for natural language processing and generation. While pitching to our news media clients, we quickly found that it pays to develop an early hypothesis on which AI applications are worth investing in.

To aid the decision process, we put together an intuitive model that places AI applications in one of four quadrants, based on how well we expect the AI use case to performs in terms of money (X-axis) and trust (Y-axis):

• No brainer — time savings and potential new revenue compensate for the initial investment and use of AI helps increase readers’ trust
• Big spender — the initial investment is higher than the monetary gain, but AI contributes to enhance trust. It pays to play the long game (with check-ins along the way), as the reputation gains indirectly lead to monetary gains.
• Risky business — these are typical low cost investments in ‘shiny new AI things’ that in the long run (or if underinvested in) pose the risk of undermining news consumers’ trust in the publisher
• Don’t, just don’t — no money + no trust = the easiest way to go bust.

Illustration by Frederique Matti

As non-scientific as this model is, we found that using it helps elevate the conversation from “will robots take our jobs?” to “how does using AI help journalists do their jobs better”?

In fact, we use this model to discuss the product development roadmap for Orbit. Nothing like eating your own dog food, so here is how we assessed three common Orbit AI use cases with our newsroom clients:

Automated news with natural language generation (NLG)

Routine reporting based on number updates (for example sports results or company financial reports) is resource intensive for the newsrooms, and boring (and sometimes error prone) for the journalists. Natural language generation (NLG) technology solves this problem by letting algorithms read real-time number updates and converting opaque data sets into reader friendly stories. Successful examples include Associated Press’ use of robots for reporting on baseball game results and company earnings, and Norwegian counterpart NTB’s football robot journalist, powered by Orbit Generator.

1. Impact on the value chain: Writing robots work at the heart of the editorial workflow, production. This explains reports of journalists skepticism that machines are likely to learn “all the nuances of human expression that help determine how events are reported”. Reservations tend to belong to the newsrooms that have yet to experiment with and benefit from the implementation of robot journalists. At NTB, journalists quickly welcomed the digital football reporter in their ranks, when they realised that it only took away the boring, repetitive job of reporting the football match results. As the robot pushes out a first draft of the football match reports without human intervention, seconds after the referee blew the final whistle, reporters gain the precious first ten minutes in the locker rooms and get their story angles by interviewing the players instead.
2. Money: According to AP, automated news writing saves up to 20% of journalists’ time. Both AP and NTB are now able to cover Minor League Baseball (and Norwegian little league football, respectively), and other events that never would have received journalists’ attention a few years ago. The newsrooms at AP and NTB are now able to produce considerably more output with the same or less amount of resources. The cost savings alone defend the implementation of artificial writers that take between 2–6 work months to develop, depending on data availability and the complexity of the machine learning models required to interpret data. Considerable cost cuts aside, natural language generation technology finally provides newsrooms with the resources to produce the right kind of personalisation. Special interest news audiences like local sports fans or shareholders in niche companies are willing to pay premium subscriptions for niche content that human journalists would not have had the bandwidth to cover.
3. Trust: To the extent that newsrooms possess quality data, robot journalists should increase news consumers’ trust in the news media brands that use artificial writers. Working with numbers is error prone, and machines are better than humans at being precise. According to AP only 1% of the articles written by machines contained errors, as opposed to 6–7% of the reports written by humans.

Quick assessment: NLG-assisted news writing is a No Brainer

News curation with natural language processing (NLP)

Uncategorised news articles barely survive their publish date. To atomise content, and be able to place a piece of news on a timeline, or surface valuable evergreens, news articles need to be labelled with metadata (content tags). The New York Times was among the first newspapers to show the benefits of tagging 15-years worth of articles when they made their 16,000 food recipe archive available through NYT cooking. NYT Cooking is also a prime example of how labour intensive retroactive manual tagging is. Not to mention journalists hate it. The fact that natural language processing algorithms can do the same job in milliseconds primes AI for content classification as an investment no-brainer.

1. Impact on the value chain. Using NLP algorithms to analyze and classify large amounts of unstructured content is probably the least sexy and the most effective and versatile use case for AI in the newsroom. Auto-tagging removes a step in the production workflow, as journalists don’t need to worry about manually tagging articles at the time of publishing in the content management system. As most newsrooms admit, getting journalists to tag content according to an internal taxonomy is about as easy as herding cats. Manual tagging by journalists at best results in incomplete (and likely biased) training data for NLP models. At Bakken & Bæck, we worked with tabloid Dagbladet to explore ways in which news publishers can use Orbit NLP to enhance the production of quality journalism on public debates. The project developed a neural network-based language model that analyses, classifies and clusters debate content by topic. The tool helps journalists do in-depth research in Dagbladet’s archive prior to publishing, and gives readers more context to the news they’re reading. The newly developed NLP applications at Dagbladet impact the entire news value chain.
2. Money: Auto-tagging undoubtedly saves time and frustration on manually organizing content. Surfacing evergreen content through distribution platforms like Blendle can be a direct source of revenue. On the ad targeting side, reliable content segmentation can hopefully both help publishers increase ad revenues and save readers unfortunate ad juxtapositions.
3. Trust: Machine bias issues aside, it is easier to trust machines with objective research or clustering content in compelling stories about how various news topics or public debates develop over time. Consistent use of natural language processing to check facts and previous research has clear potential to enhance readers’ trust in the news media that employs it.

Quick assessment: NLP-assisted news curation is a Big $pender

Bots, bots, bots

Let’s face it, newsbots today are pale versions of the artificially intelligent conversation partners that can provide us with a sensible and personally relevant news update. At best, newsbots solve the need to reach news consumers where they are — on their phones, texting. Successful examples almost always include a human at the other end of the line — like NYT’s Sam Manchester reporting from the 2016 Olympic games in Rio, or the movie recommender And Chill.

1. Impact on the value chain: Newsbots live inside one of the most valuable news distribution channels and have the potential to fundamentally change the mobile interaction between news brands and consumers. Although existing bots are either rudimentary forms of keyword search navigation or glorified mass push notifications, news publishers should not give up exploring conversational news consumption. Gartner predicts that 30% of web browsing will be done via screen-less user interfaces (so called Zero-UI) by 2020. Today’s awkward ‘botification of news’ could be the stepping stone to voice-interactions that facilitate the coveted personal relationship between news organizations and their readers.
2. Money: Beyond the referrer value of bots that may or may not drive traffic to the news outlets’ own sites, the monetization of bots is as of now a large unknown. Based on the investments required to develop a half-decent customer service bot, a smart newsbot investment can throw back a newsroom by at least 6 development months.
3. Trust: Two words: Microsoft Tay. Building a newsbot that facilitates trust through two-way conversations between news consumers and publishers is not impossible, but it first requires some heavy lifting in natural language processing for understanding consumers’ queries, and in natural language generation for producing personalized responses. According to Nieman, with the “print world rapidly disappearing, the low actual cost of digital news delivery revolutionizes the business model of the 2020s — that is, if publishers can produce sufficient valued content at scale.” Bots may be the key to scale and personalize content distribution, but building trusted newsbots truly requires a commitment to the long-game and high investments in ‘strong AI’.

Quick assessment: Don’t, just don’t (for now)

Finally, can AI fix fake news?

It is tempting — and probably not far from the truth — to believe that any problem can be solved with machine learning, as long as the problem is formulated correctly, and fed enough training data and modelling competence. Several digital methods to trace the production, circulation and reception of fake news online, most of which are aptly captured in the excellent Field Guide to Fake News, can benefit from machine learning capabilities.

Profiling fake news sources is a particularly interesting AI challenge, but getting consistent training data sets for this problem requires complex tracking mechanisms that resemble the hairy attribution problem in marketing. To draw the parallel further, the clue to solving the fake news problem is solving the online-offline attribution. In the fake news context, this means understanding why fake news is created in the first place, what happens with readers’ opinions after consuming fake news, and why some choose to propagate known ‘alternative facts’.

Using misinformation mechanisms as a means to undermine democracy is not a new phenomenon. News publishers will benefit more from refraining from extending band-aid solutionism to AI, and instead focus on using machine learning to do better research and actively increase news integrity. The aforementioned topic extraction using neural network models for natural language processing is a good example of how AI can help initiatives like Faktisk.no, WikiTribune and other standalone site dedicated entirely to fact-checking.

No luck? Get some

AI alone does not make good journalism, but machines in the newsroom can give journalists the luxury of time and powerful tools to find and tell good stories. To afford this luxury, newsrooms need to solve the very resource hurdle AI promises to fix.

The clue to this money catch-22 is to take a portfolio approach to bringing AI in the newsroom. News publishers need to move AI investments from isolated proofs of concept to AI applications that work side by side with the journalists in the most resource-intensive parts of the editorial workflow.

Most importantly, newsrooms need to carve out their own business case for AI, and ensure that automation both makes them the money to survive, and rebuilds consumers’ trust in the news. The only sustainable way to build an AI-powered newsroom is to consciously develop AI applications that enhance trust and increase journalists’ ability to digest, decode and present a news picture that accurately reflects the world we live in.

Disclaimer: The author of this blog post is a business developer at Orbit.ai — a Bakken & Bæck venture that develops AI tools for natural language text automation. Any resemblance between our products and the AI tools we think newsrooms should invest in, is definitely not coincidental. If we didn’t think this stuff worked, we would not spend our time making it.