Tow Fellows Brian Abelson, Stijn Debrouwere, and Michael Keller to Study the Impact of Journalism
Over the past couple of years, a veritable movement has arisen around measuring the impact of journalism. This movement seeks to find alternatives to the widely-used metrics of page views, time on page, and social media shares, none of which fully capture the impact of a news story. Representatives of the Washington Post, The New York Times, and ProPublica have all openly discussed this need.
Through these discussions, it’s become clear that newsrooms need a better way to measure their role in the world. It’s what Tow Fellow Jonathan Stray calls the last mile problem: once it’s published, “how does journalism actually reach people?” But while some newsrooms have in-house workflows for measuring impact and others are integrating tools into their CMS, there is not yet for comprehensive tool for capturing and comparing the quantitative and qualitative impact of news stories.
Over the next year, we’re embarking on a Tow Center research project to investigate what newsrooms see as useful measures of impact and what a helpful “impact measurement platform” might look like. To do this, we’re building a platform over the next few months that will help newsrooms measure, record, and report relevant events they consider “impactful.” We’re calling it NewsLynx. Once its ready for launch, we’ll give it to interested newsrooms and observe:
1. How they measure impact, e.g. what kinds of taxonomies they tend toward, what events they consider most impactful.
2. How this kind of platform helps newsrooms fulfill their institutional goals.
Research sample
As in any research project, our sample is extremely important. While the platform will be open to anyone (just fill out this survey and select “Yes” to the last question), we’re working closely with the Investigative News Network, a collection of 90+ non-profit investigative newsrooms. These newsrooms are largely donor-funded and thus have a group of stakeholders vested in the results of the journalism. Many already have people who are tasked, at least part-time, with this type of impact collection. Investigative reporting also has more saliently measurable effects, e.g. laws changed etc, than other types of journalism, so we think it’s a good place to start.
How do we think of impact
We’re splitting impact into two main categories of measurement: quantitative and qualitative.
- Quantitative Newsrooms have access to a lot of numbers through multiple analytic services; we don’t intend to reinvent the wheel here. We do think, however, that these myriad data sources can be better organized to provide for easier interpretation. There are also some simple social metrics, like shares over time, which aren’t currently being captured. For this, we’re building off of the open-source work of our team member Stijn Debrouwere.
- Qualitative In our minds, the more interesting facets of impact are qualitative measures, such as ensuing investigations, references in official reports, or legislative actions.These are also the events that are the most difficult to capture. We’re planning on building tools to reduce the difficulty of recording such events and associating them with a given article or series of articles. As we work with our partner newsrooms, we’ll incorporate their feedback to add features and refine the user experience.
Taxonomies
One of the most challenging aspects of qualitative measurement is constructing taxonomies that enable comparisons across many different contexts. As anyone who has attempted to create one knows, taxonomies are tough because the world is messy and cannot be reduced into neat buckets. Take, for instance, Jorge Luis Borges’s Celestial Emporium of Benevolent Knowledge which divided animals into the following categories:
- Belonging to the emperor
- Embalmed
- Tame
- Suckling pigs
- Sirens
- Fabulous
- Stray dogs
- Included in the present classification
- Frenzied
- Innumerable
- Drawn with a very fine camelhair brush
- Et cetera
- Having just broken the water pitcher
- That from a long way off look like flies.
Working with a number of other organizations we’re hoping to build an impact vocabulary that will give newsrooms a useful and standardized framework which also permits customization. This type of flexible taxonomy is an example of how our research will try to theorize solutions while testing and observing how they work in practice.
Preliminary feature list
While we’re just getting started, here’s a sense of what NewsLynx will include:
- Tracking of social media “mentions” and “likes” over time on Twitter and Facebook.
- Tracking of mentions by lists of people, e.g. local and national representatives, other journalists, or institutional representatives.
- Integration with Google Analytics and other metric providers.
- A Google Alert-like river of mentions that can be approved and associated with a given article.
- A Customizable qualitative taxonomy and tagging system.
- An interface for recording “impact” events not tied to automatic processes.
- “If-This-Then-Impact” recipes for custom combinations of events that should trigger an event to be recorded.
- A report generator for distributing impact assessments to staff, board members, and financial backers.
Next steps
We’ve been conducting initial research for a few months and we’re now broadening our scope. By filling out our survey you can help us better determine what questions to ask in our research and which features to add to our platform (if you would like to use NewsLynx the only thing you have to do is provide contact details in the final section of this survey). Over the coming months, we’ll continue to update our progress on this blog and our initial results.
If you’d like to get in touch with us, our email is contact@newslynx.org, or you can sign up for updates at newslynx.org.
Brian Abelson, Stijn Debrouwere, and Michael Keller are Tow Fellow working on the Tow Center’s NewsLynx project at the Tow Center for Digital Journalism. The NewsLynx project is a project made possible by generous funding from both The Tow Foundation and the John S. and James L. Knight Foundation. The NewsLynx project seeks to create an platform for media organizations to document the short and long-term impact of investigative news stories by combining automated metrics with flexible tools for capturing qualitative insights. Follow Brian Abelson on Twitter @brianabelson, Stijn Debrouwere on Twitter @stdbrouw, and follow Michael Keller on Twitter @mhkeller To learn more about the Tow Center Fellowship Program, please contact the Tow Center’s Research Director Taylor Owen: taylor.owen@columbia.edu.
