Engaged journalism has measurable benefits.
The latest Agora Journalism Center Report suggests that newsrooms using Hearken to practice “public-powered journalism” are finding some measurable success.
--
At every single conference, workshop, and convening focused on engagement journalism that we have organized or attended over the past two years, and in our lightning chats and other conversations on Gather, one question repeatedly gets raised: How do we make the case for investing in engagement? More specifically, the question is often framed as, “How do I convince my boss/manager/editor that doing engagement is worth it?”
“Engagement” is an evolving set of practices within journalism, and its impact on attracting, developing and satisfying audiences has yet to be fully and rigorously documented, particularly by the scholarly community. One entity that is making a strong case for the commercial as well as the journalistic value of doing engagement work is Hearken, whose landing page features in bold letters: “Does Hearken Work? Yes.”
Over the past several months we’ve had the opportunity to interview reporters and editors from 15 newsrooms across the country that have been using Hearken’s tools and consulting. We focused on outlets that are using Hearken because of its growing presence in newsrooms (Hearken now contracts with about 130 media organizations in the US and abroad) and because at conferences and professional workshops focused on journalism and the news industry, the Hearken model is regularly invoked as a promising approach to engaged journalism that also contributes to the bottom line.
Hearken’s claim that “public-powered journalism” can not only deepen news outlets’ relationships with audiences and make news more relevant, but also help the news industry’s troubled business model, has received some scholarly scrutiny. Jacob Nelson’s ethnographic case study of the company concluded that engagement remains an “elusive metric” because the operational definition of engagement has not been settled, and because the empirical case for the economic benefits of Hearken-style engagement has not been made. Though Nelson’s study stirred some controversy and some push-back from Hearken itself, his basic insight remains relevant: “As publications struggle to survive, journalists can’t help but think that improving the relationship between news producers and the audience is at least a step in the right direction.”
Our study — available in full here — takes a different tack. We set out to examine how newsrooms are deploying the Hearken model — not just its tools for audience-submitted news topics and public voting rounds on those topics, but also its methodology and philosophy of public-powered journalism. As fellow travelers in the field of engaged journalism, we have admired Hearken’s work, and we also have questions about how resource-strapped newsrooms are implementing its journalistic approach, and with what impact on organizational culture, workflow and attitudes.
Our research, based on interviews with journalists using Hearken, does not directly examine whether Hearken has improved those organizations’ bottom line. That would require a different methodology and access to sensitive data. But our interviews do shed light on how outlets are measuring the impact of doing engagement — and finding successes.
Newsroom engagement metrics
The newsrooms we spoke with employ various metrics, both quantitative and qualitative, to assess the impact of using Hearken, and they all reported some degree of success in using Hearken to improve audience engagement and/or their news product. (That “and/or” is important, as we’ll see in a moment).
The mix of quantitative and qualitative measures span four dimensions of engagement: reach, engaged time, quality (of individual stories and of overall performance) and conversion of engagement into memberships or subscriptions. Quantitative metrics include conventional measures such as clicks and downloads, plus Hearken-specific measures such as the number of audience questions submitted and the amount of participation in voting rounds. These newsrooms also rely on qualitative assessments of quality of questions they receive from the public, the quality of audience feedback and comments on Hearken-driven stories, as well as journalists’ own personal satisfaction and pride in their work.
Overall, our interviews indicate that nearly every newsroom in this sample saw at least some success in using Hearken tools by at least one of these measures.
Reach, engaged time & conversions
In terms of reach, newsrooms closely track how Hearken-driven stories compare to typical news stories, mostly by looking at internal rankings of stories according to the number of people clicking on the web version or downloading a podcast. WUWM in Milwaukee reports that their Hearken-driven Bubbler Talk stories typically make it into the station’s top five of most-listened-to stories. Dustin Leed, digital editor at LancasterOnline, which uses Hearken to produce its We the People series, told us that“Every one of our Hearken stories is in the top 50 posts of the year for engagement and time spent on the story.”
For many newsrooms in our sample, the time that audience members spend actually reading, viewing or listening to Hearken-driven stories is a key indicator of success. Angela Evancie, managing editor and host of the Brave Little State podcast at Vermont Public Radio, told us that “our [Hearken-driven] web posts are some of the most engaging that we put online…[The] number of minutes that people spend looking at any given post [compared to others] are on average quite high.” More and longer engagement is also a trend that the Texas Tribune has seen. “The bounce rate on the Texplainer stories are much lower,” Amanda Zamora, the Tribune’schief audience officer, told us. “The time spent on those stories is higher, and we’re bringing back more repeat visitors through those pages.” For LancasterOnline’s Hearken-driven stories, compared to conventional stories, Dustin Leed told us “the time spent in engagement, it’s always through the roof. People read it a long time. It’s not like a breaking news story where they’re going to read the lead and a paragraph and then boom, leave.”
The most ambitious goal for using Hearken is to convert initial interactions with question askers into lasting relationships that yield memberships or subscriptions. “Engagement, for us,” said Zamora of the Texas Tribune, “means moving people from that initial contact into a more engaged experience with us, which primarily means they’re signing up for our newsletters, they are attending our events, they are, in some way, expressing an interest in having a persistent relationship with us.”
While Hearken promotes its tools in part by claiming that its model helps to attract more paying subscribers, only a few news organizations we spoke with had implemented a structured approach to converting question-askers into members or subscribers. At the time of our interview, the Tribune was working on developing a conversion metric to measure how many question askers become subscribers. Vermont Public Radio hadn’t developed such a specific approach yet but planned to better connect Hearken-driven interactions to membership recruitment.
Quality measures
In addition to quantitative measures, these newsrooms also look at qualitative ways to assess the impact of Hearken-driven stories. Elizabeth Koekenga-Whitmire, senior director of audience at AL.com, for example, told us that audience comments are also a good indicator of quality. “We…spend a lot of time, looking at the comments on the site, and the comments on Facebook when we share [a Hearken-driven story],” she said. And some newsrooms reported a sense of pride in their engagement-driven work that also serves, for them, as a measure of success. Olivia Allen-Price, producing manager and host of KQED’s Bay Curious in San Francisco looks toward “more of a squishy interpretation of how successful a story is, and that combination of, how happy were we as journalists with the quality of the content that was in the story? ….Was the story compelling?”
Overall, our interviews indicate some degree of success at each of these newsrooms in using Hearken, measured across these various metrics. The metrics are not uniform across newsrooms, however, and most newsrooms hadn’t developed clear metrics to address the crucial question of conversion.
Attitudes toward the public: An under-examined metric?
We also asked each journalist we interviewed whether working with Hearken had changed their expectations of and attitudes about their audiences. We asked this because, at its heart, the Hearken approach advocates that journalists re-examine their perceptions of the public. Embracing the idea of the audience as participants in news-making is perhaps the most fundamental way that Hearken and other “relational” engagement approaches challenge established ways of doing journalism.
The responses we heard fell into two groups. One group, mostly editors, emphasized that Hearken did not change their expectations or attitudes because they had believed in the value of engaging with the audience in the first place. For this group, using Hearken reinforced their commitment to engagement and, for some, has empowered them to promote a culture shift in their newsrooms. “I drank the Kool-Aid on this approach to journalism a while ago,” said Amanda Zamora at the Texas Tribune. Using Hearken, she said, “is helping me influence a culture that existed without an understanding of what I meant by engaged journalism when I walked in the door. People understand through seeing it in practice.” Elizabeth Koekenga-Whitmire at AL.com said, “I don’t think that using Hearken has changed our expectations. I think we’ve always had really high expectations, and we’ve always kind of gone to the audience to help drive the direction that we’re going in. I think what Hearken has done is give us a really good system for that…”
The other group, mostly reporters, described changes, sometimes after initial skepticism, toward embracing deeper engagement with audience members. “I like that it introduced us to bringing the listener, the community, into the reporting process a lot earlier, [helping us rethink] the way that we decide what our coverage is,“ said Michelle Maternowski (WUWM, Milwaukee). As Laura Ellis (WFPL, Gainesville) put it, “I think it definitely makes us think about who we’re involving in a story. There’s always this cliché of, ‘Let’s find a real person who’s going to be affected by this story and see what they’re saying.’ I think that at least for me, I think more about how can this person shape the story? How can this person’s experience or curiosity determine things about how this story ends up sounding?” Ann-Elise Henzl (WUWM, Milwaukee) said that when they began using the Hearken voting rounds to choose story topics, she remembered thinking, “‘Oh, come on, we’re going really turn over control to listeners?’ Because… I had no idea what people would be interested in. Maybe I just didn’t respect how constructive their ideas would be, or how creative, and how interested they are in things that are around them.”
Some of these journalists also found it rewarding to interact more directly and frequently with audience members because it boosted their own morale. “I realized that the audience is so much more engaged than I thought that they were,” said Christina Morales (WUFT, Gainesville). “That was very exciting for me to see. People actually cared about what I was putting out there, and that my material was actually really good.” And several also said that audience-submitted questions helped them develop a better sense for important issues that might otherwise be overlooked. “I think it’s made me, as a journalist, think about how to use audience engagement in other facets of my reporting,” said Ashlie Stevens (WFPL, Louisville). “I think that maybe sometimes what we need to realize as reporters…that the people who are actually living it have questions that we may not even think about.” As Katherine Nagasawa (WBEZ, Chicago) put it, “I definitely think it’s like a very organized way of tapping into the public’s conscious, in a way, what’s on their mind.”
Though it certainly needs more research, these findings suggest an under-examined set of metrics for measuring the impact of engagement — impact, that is, on journalists themselves and the newsrooms they work within. If practicing deeper community engagement improves journalists’ satisfaction with their own work and positively alters their perceptions and understanding of the communities they serve, this could be one pathway toward rebuilding trust between the media and the public.
We’d like to hear from you about our report, which you can read in full here, and about how your organization is becoming engagement-ready. You can share your thoughts by writing a response below or emailing me.
— Regina Lawrence
Regina G. Lawrence is professor and associate dean for Portland at the University of Oregon School of Journalism and Communication, and director of the Agora Journalism Center, the gathering place for innovation in communication and civic engagement. She can be reached at rgl@uoregon.edu.
