Measuring Beyond the Page View
How the Media Impact Project Attempts to Wean Newsrooms Off Simplistic Metrics
Many news organizations live and breathe page views, but every time a newsroom drops this outdated, surface-level metrics mindset, a data analyst gets its wings.
It’s not just that measuring a media outlet’s success by the number of page views it gets is outdated — it’s downright misguided. This measurement problem ends up skewing the industry toward metrics that aren’t meaningful. But figuring out exactly what should be measured isn’t easy, either. For the past two years, the Media Impact Project has been waging a Sisyphean battle to convince newsrooms to kick these old habits and adopt a new way of looking at what they do.
Funded by both the Bill & Melinda Gates and the John S. and James L. Knight foundations, the MIP is a data-analysis hub at the USC Annenberg Norman Lear Center that’s trying to develop a more nuanced approach to measuring the impact of media. It partners with news organizations to help them better understand their audience, and, rather than going on intuition or a hunch (not uncommon in the industry), the MIP encourages newsrooms to embrace data as the basis for all of the decisions they make.
That’s become increasingly essential in remaining relevant and profitable, and people like Bill Gates know this. The Gates Foundation invests heavily in news organizations like ProPublica, the Pacific News Service and Guardian News because it want to see more coverage of the issues it believes are important and under-covered, like global health and development, education reform and beyond.
But they also want a better return on investment. These foundations need to know that every dollar they spend is contributing toward their social impact goals, and data will tell them whether or not they are. But first, organizations need someone to show them how to analyze that data. Which brings us back to the MIP.
“The organizations we work with are early adopters who say they really want to know what the data is telling them,” says Dana Chinn, Director of the MIP. “Part of our process is determining whether they really mean it or not.”
Usually, they find out pretty fast. Chinn and other data analysts embed themselves in each newsroom’s culture to work with editors and staff to see what’s working — and what’s not — when it comes to audience engagement and increasing readership. Those are scary conversations for editors to have. (One editor, upon learning the implications of some of her platform’s analytics, refused to let them be seen and discussed in a meeting.) And while each organization the MIP works with is different, their problems usually stem from the same issue.
“There are standardized metrics everyone uses,” Chinn says — those pesky page views, among other measuring sticks that don’t mean a lot on their own. “But there are no simple answers anymore, because truly measuring the impact of a story on an audience isn’t simple. As metrics change over time, so should the way we measure them.”
That’s not exactly what editors want to hear. It’s difficult enough having to face the reality that the methods they’ve been using to measure success are faulty and incomplete at best. Most of them want a quick fix, and when they find out the MIP can’t offer one, they sometimes become frustrated. After all, page views are often good enough for advertisers.
In his tell-all book, Trust Me, I’m Lying, reformed media manipulator Ryan Holiday sums up the common mindset well: “Publishers and advertisers can’t differentiate between the types of impressions an ad does on a site.” He goes on: “A perusing reader is no better than an accidental reader. An article that provides worthwhile advice is no more valuable than one instantly forgotten.”
Bottom line: a click is a click.
Some perfectly noble organizations will dispute these claims, but Holiday makes a point when he asserts that there’s a kind of cluelessness when it comes to gauging the impressions of an ad — or a story — on a site. After all, a major source of income for news sites is advertising, and newsrooms like to tout page views to advertisers when the number is high (or otherwise scramble to create content that will garner more clicks if the number is low).
The further the page view myth is debunked, however, the less this oversimplified understanding of metrics will cut it for advertisers, because they’re becoming savvier when it comes to understanding metrics. That means they’re increasingly expecting more in-depth analysis. They want news sites to know their audiences inside and out.
Chinn cites Univision as one organization that’s tuned in to who its audience is and what content is most valuable to that audience. As a result, Univision’s newsroom is receptive to — and unafraid of — the MIP’s analysis.
For most other organizations that are less familiar with their core audience, it can be more of a challenge to be coaxed away from the metrics they cling to, and that’s why Chinn says establishing trust with the newsroom staff is crucial. Only then can they delve into the nitty-gritty of a newsroom’s numbers.
“It’s a consultant’s job to talk about the uncomfortable stuff and what’s going to be good for the client,” Chinn says. And usually, it’s not so scary. One of the most basic yet crucial stats the MIP looks at is the number of visits, which differs from page views in that a visit could consist of five minutes, 10 clicks and one comment, or five seconds and no clicks or comments. From there, the next most important metric to look for is returning visitors, and that’s where the good stuff comes from. It’s the returning visitors whose demographics newsrooms should be most concerned with, and the MIP walks them through how to analyze those figures.
In doing so, newsrooms are led to clearer correlations between returning readers and the content that has lasting power, and editors can then fine-tune their editorial direction and focus to cater to that audience as needed. From there, the numbers usually get better. When they know exactly who their most loyal visitors are, newsrooms are emboldened to produce more incisive content, and that attracts dollars.
With each organization the MIP partners with, the more information it has to base its predictions for the future on — after all, metrics are always evolving. “There’s no big reveal when we’re working with the operations of a company, so it’s not like we leave them with a big package,” says Chinn. But the MIP defines its success as leaving an organization more adept at understanding these complicated measures and not being afraid to dig into their meaning.
And for Chinn, “sometimes it’s just someone saying, ‘Now I understand why page views aren’t the most important,’ and to me that is a huge step.”
