Metrics and the media: we can measure it — but can we manage it?
In October I chaired the ‘Data Strategy’ track of talks at the Monetising Media conference: individuals in every part of the industry talking about how metrics now inform not just content strategy but revenue, advertising, and customer relations.
As I introduced the day I was thinking about two pieces of data in particular: research by the Tow Center’s Caitlin Petre into the use of Chartbeat; and Checking, Sharing, Clicking and Linking,a piece of research into consumption.
Gawker and the New York Times: good intentions
Caitlin Petre’s The Traffic Factories tells the insider story of how analytics were used at Gawker and the New York Times, two polar extremes when it comes to metrics: historically The New York Times trusted their reporters too little to interpret web analytics, while Gawker perhaps trusted theirs too much.
What emerges from The Traffic Factories is not that analytics are ‘good’ or ‘bad’, but rather that individual journalists can be — at least when it comes to knowing what to do with that data.
At Gawker, for example, journalists treated metrics like a game, chasing high scores and dopamine hits rather than stories or a meaningful relationship with audience:
“Some staffers made explicit analogies to drugs when discussing metrics. [One writer] admitted to having been a “Chartbeat addict” at previous digital media jobs. He said he was trying to limit his exposure to the dashboard, with limited success: “At Gawker Media it’s like I’m a cocaine addict on vacation in Colombia.” Others compared the perpetual hunt for traffic to playing a game or gambling.
As a result the company had to change tack somewhat, taking their notorious dashboard screen out of the newsroom and curating an editor’s ‘best of’ blog.
At the New York Times reporters would argue about the ‘most emailed’ list, find ways to access other analytics regardless of company policy, and the organisation has now opened them up.
But the same thing has happened: editors have decided to more proactively showcase their reporters’ best work.
In both cases the companies had the best of intentions — to empower journalists and protect them respectively — but journalists did not do what was anticipated, and editors realised they had to show leadership in making their own metrics explicit.
16 types of consumption
I wrote about Checking, Sharing, Clicking and Linking in one of the most-read posts on the Online Journalism Blog last year (yes, I check my metrics closely too). It was a timely reminder of a basic principle that every student learns in research methodology: no data collection method is perfect, and a combination of methods often works best.
In this case a piece of qualitative data collection highlighted an analytics blind spot: people still ‘read’ a story even if they never visit the page it is published on.
How? Well we ‘scan‘ and we ‘snack‘; we ‘check‘ and we ‘monitor‘ using headlines on homepages and social media alone. Some of these behaviours support our social life; some our role as citizens and voters.
They are valuable, therefore — but unmeasured by mass market analytics tools.
What gets measured gets managed
The classic quote has it that:
“What gets measured gets managed — even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so.”
If we don’t invest in the right metrics, we make bad decisions. If we outsource the decision on what to measure to the wrong company, they make bad decisions too.
And it’s not just what gets measured: it’s how well people have been trained to manage that information. A journalist trained only to treat metrics as a leaderboard is a bad manager of their own content.
So when we delegate management responsibility we should make sure the journalists are trained to handle that. In fact, this is among the conclusions of The Traffic Factories:
“Most journalists are too busy with their daily assignments to think extensively or abstractly about the role of metrics in their organization, or which metrics best complement their journalistic goals. As a result, they tend to consult, interpret, and use metrics in an ad hoc way. But this data is simply too powerful to implement on the fly. Newsrooms should create opportunities — whether internally or by partnering with outside researchers — for reflective, deliberate thinking removed from daily production pressures about how best to use analytics.”
Those opportunities should exist in journalism education, too.
For students on my MA in Online Journalism, for example, measuring impact is the very first thing they do (yes, before they even write a story). There are 3 key reasons for this: a sense of audience; clarity about objectives; and taking control of the process rather than it taking control of you.
Because like an interview with any source, a journalist should know to always ask one critical question of data: “What are you not telling me?”
This post was first published on the Online Journalism Blog, where you can subscribe to receive email updates.
