Are robots on their way to local newsrooms?

The RADAR project is a joint venture between the Press Association and data journalism start-up Urbs Media. It uses automation to deliver data-driven local news stories. RADAR editor Gary Rogers explains what the project is trying to achieve and how it might be coming to your newsroom soon.

Type ‘robot journalist’ into Google and people are already predicting a Terminator-style future

AI seems to be a bit of a buzz phrase in the media of late. Countless articles discuss whether Artificial Intelligence will be a saviour for news or the final nail in a coffin as journalists are replaced by robots?

If you’ve read any of those AI stories then you’ve probably noticed the picture of the robot hands on the keyboard that accompanies most of them.

That robot has been following us around the internet, appearing whenever anyone writes anything about RADAR. But they are using the wrong picture as RADAR may be a news automation project but it’s human hands on the keyboard in our newsroom.

Around 50 newsrooms around the UK are familiar with RADAR as they have been part of our pilot group. For others I’d best track back to the start to explain what we are trying to achieve.

RADAR is a joint operation between the Press Association and data-journalism start up Urbs Media. Both Urbs founders started our news careers in local press, and we wanted to try some fresh ideas.

We started up three years ago by trying to tell the story of a single city through its data and found a strong flow of punchy and informative copy.

Initially, we thought we might compete with traditional local media, but the more we experimented, and the more we spoke to publishers across the spectrum, it seemed clear that we could add more value by helping established newsrooms. We quickly morphed from a news outlet to a new form of lean, tech-based news agency, supplying newspapers and broadcasters with data driven stories.

Our friends at the PA also saw the potential to serve all their regional and local news clients with a stream of highly localised stories based on data.

A front page made possible thanks to RADAR’s journalism

The challenge was to find a way to do this efficiently at scale. Could you do it once for everyone?

We turned to a technology called Natural Language Generation. NLG has been around for a little while. It’s been used to write product descriptions, sales reports, or anything where a lot of numbers need to be translated into meaningful text. Its use in journalism has been quite limited, though several organisations have used if for match reports in sport, or financial news.

Sport and finance had been obvious targets as they both have well structured data. But we saw that a lot of the UK’s open data was neatly structured by geography, most often by local authorities. So instead of lots of football results in a sheet of data we had lots of figures from each local authority on a huge array of topics — health, transport, crime, education, work, the environment — the core beats of local news.

Urbs and the PA took the idea of covering local news topics via NLG to Google’s Digital News Initiative last year and they gave us major funding to develop the idea. RADAR was born, and work began last autumn.

There’s perhaps an irony to the fact that the first thing we did in a project that was being characterised as a rise of the robot journalists was to recruit some reporters, human reporters. So far we have hired a team of three, two of whom learnt their trade at local titles.

Elsewhere, NLG technology has been put to work by editorial teams working with developers to write the complex story templates.

A RADAR story on the front of the Ham and High

We think this misses out on the potential of the unique skills reporters possess. Instead, our ambition was to make NLG part of the journalist’s toolkit, a way to write a story — not the work of a coder.

There are several different NLG software packages on the market. We work with Arria as we found their tool, NLG Studio, well-suited to the task and user friendly for journalists. That’s not to say there’s not been a steep learning curve.

The first challenge is to remember that you are not writing one story but maybe hundreds, perhaps thousands. The data drives each version of the story, and every sentence written in an NLG story must work no matter what the numbers say. And yes, it does sometimes make your head hurt.

We spent the early weeks of the project developing a way of working, from selection of the data, to uncovering the story and then crafting a template which uses the best angles and the right numbers in each version.

It was clear from the start that we couldn’t develop this service alone. We needed help from our potential customers.

Through the relationships that already existed with the PA and Urbs we were able to build a pilot group that included all the major local news publishers in the UK, plus some independents and hyperlocals. Overall, our pilot publishers represent over 1,000 titles.

We asked each publisher to suggest a handful of titles that combined would give us a spread across the whole country including dailies, weeklies and online only publications.

RADAR content on the Wolverhampton Express and Star website

It amounted to around 50 titles and we began sending them stories at the end of November. Within a few hours, the Wolverhampton Express and Star had published the world’s first NLG-powered local news story.

Within a few weeks a page lead in the Dover Express was on show at an IBM developer conference in San Francisco, and the Bournemouth Echo was the subject of a Bloomberg Technology podcast. More importantly, local audiences were getting important stories that they might not otherwise have read.

We are indebted to the nominated ‘gatekeeper’ for each company who have guided our thinking, distributed our copy and sent us feedback. The enthusiasm and depth of insight has been fantastic and has really helped us fine tune the product and service so that it is gradually meshing with newsroom workflows.

This Dover Express lead was highlighted at a tech conference in San Fransisco

We deliver stories that are ready to publish. Many contain quotes and context at a national level. The newsrooms receiving the copy can choose to publish as is, or build a stronger story with local context, reaction or calls to action. We have been delighted to see the copy used in both ways.

To date we have sent out around 60 stories with multiple versions of each. Hundreds of RADAR stories have been published.

Usage has ranged from a few pars to front page splashes, with regular page leads and some great inside spreads based on our work.

We hope that RADAR content adds value in a few ways:

· It will offer a regular daily supply of local stories to bolster the efforts of each newsroom

· It delivers stories that might have gone un-noticed or would not be possible to tackle in smaller newsrooms

· It will save journalists time by doing the grunt work in the data and delivering the results in easy to use copy

· It can free up reporters from the daily churn and allow them to chase other stories

Limiting the pilot group has meant a stronger feedback loop that has helped guide our work. It was also necessary as part of the Google funding was dedicated to building a distribution system that offers the relevant stories to each title.

Work on that is progressing well and in the next few weeks we’ll start using a prototype version in place of the current process of sending out batches of stories by email.

That means expansion. With a targeted delivery system, we can start to serve the whole UK instead of just the parts relevant to our pilot group titles. We’ll be opening RADAR up on a trial basis for the whole country. We hope that your newsroom will be signing up.

  • Like this? Sign up to our weekly email covering interesting stuff happening in local media in the UK — click here to get the email