Designing better digital metrics in The Times and The Sunday Times newsrooms

See here for our latest post our ongoing adventures with metrics in the newsroom.

At The Times and The Sunday Times, like many publishers, we work hard to try and make it easier for our newsrooms to use data to make better decisions.

However, traditional digital analytics tools have fallen far short in helping us understand how our content is performing on our website and apps. The metrics these tools offer — average dwell time, unique readers, page views and the like— typically fail to provide meaningful insight, or can fundamentally mislead us about whether content is exceeding our benchmarks or clearly not meeting the needs of our readers.

Over the past few years we have developed INCA (Intelligent Newsroom Contextual Analytics), a tool designed to be accessible and actionable for our newsrooms, and that attempts to quantify how well our content is performing digitally (see here for more background on INCA).

A key aspect of INCA’s development has been the decisions we’ve made about which metrics we display and in what format. Instead of traditional metrics, INCA displays a set of indices. These indices can be used to help evaluate how well an article is performing for a given metric (e.g. dwell time or number of readers) relative to how well we expected it to perform given its context (e.g. length, position in the edition etc).

As Nick Petrie, Deputy Head of Digital for The Times and The Sunday Times puts it:

“INCA indices help augment editorial judgment to inform intelligent decisions using data, from identifying headlines or images that need modifying to commissioning new content. Ultimately it’s about reducing the time it takes for our journalists to get meaningful insights.”

Are longer articles better than shorter articles?

A metric such as average dwell time (for the uninitiated, this measures the average number of seconds a reader on our website or apps spends on an article before navigating to the next one) tends to tell us as much about the length of the article as anything else.

The chart below plots the average dwell time for individual articles against their length (number of words). Whilst we see a lot of variation in the data (we’ll come back to that later), longer articles tend to have longer average dwell…