Commonplace Forum
Published in

Commonplace Forum

Finding the gems hidden in the rough between theory and practice.

Studying Emerging Engagement

By Thomas R. Schmidt

These questions sparked a controversy last week, pitting an academic researcher against prominent proponents of new approaches to audience engagement. Jacob Nelson, a PhD candidate at Northwestern University, summarized results from his research about the community engagement tool Hearken in the Columbia Journalism Review and argued that “the audience engagement industry struggles with measuring success.” In a letter to the editor, Hearken founder and CEO Jennifer Brandel questioned Nelson’s findings and wrote they were “based on outdated research from early 2017.” Additionally, Brandel posted a rebuke on Medium, pointing to examples and evidence of Hearken’s positive impact on newsrooms. A lively debate on social media ensued.

For us at the Agora Journalism Center, this debate goes beyond the specifics of one particular study. We deeply care both about the community of practice — including the many projects represented on the Gather platform — and the value of academic research. For us, this debate raises some important issues of examining engagement in today’s journalism, the methods of social science research, and the relationship between theory and practice.

We believe that Nelson’s piece should not be characterized as a “hit job,” as some critics argued. At the same time, we also believe that the debate about “measuring success” calls for a broader conversation about how we define empirical evidence. Empirical, to pick a definition from Merriam-Webster, means “originating or based on observation or experience.” Against this backdrop, Hearken can point to promising data that is more than circumstantial and anecdotal, though not all of it meets academic standards of systematic evaluation. We don’t yet have enough studies that would meet quantitative researchers’ demand for “validity” nor qualitative researchers’ demand for “trustworthiness.” More fundamentally, however, what this debate really illustrates is the difficulty of analyzing an emerging practice in an early stage of development and the challenge of quantifying it. Fortunately, social science offers a wide range of options. For example, systematic, qualitative research can play a crucial role in helping us understand how engagement works and what it can do for journalism.

And we’re eager to make a contribution. Earlier this year we asked 28 journalists from 15 newsrooms across the country about their use of Hearken. We wanted to find out how the same tool is used in a variety of newsrooms such as public radio stations, newspapers and digital-only news outlets. While we’re still analyzing the material from these conversations, we can share a few insights that might add some texture to this debate about measuring the success of audience engagement. The full analysis will become available as an Agora Report later this year.

Editors and journalists told us that metrics play an important part in assessing how Hearken-driven stories are performing. “We’re so new and I want to be sure that is this actually working,” Angela Evancie from Vermont Public Radio said. Generally, editors pay close attention to Google Analytics and quantitative data. Some of them are also actively seeking to convert Hearken question askers into subscribers or members, i.e. monetizing engagement.

But beyond metrics, newsrooms are also monitoring more self-designed markers of success. Journalists reported that they were interested in the originality of incoming questions, the variety of questions and whether questions come from a diversity of people. Many of them think that audience engagement is a meaningful way to serve their audience. “We believe in lasting and deeper engagement, so we don’t chase clicks as much as we try to foster conversation that’s meaningful,” said Caitlin Shamberg from KCRW in Los Angeles.

Our research also holds insights about the biggest challenges for making Hearken work. A recurring issue that was identified by journalists across the board was achieving consistency in terms of staffing, resources and frequency of queries. Using Hearken is one thing, but it is yet another to build infrastructure around it for sustained engagement with the audience. That approach requires sufficient funding.

Moreover, audience engagement challenges the organizational culture of newsrooms. Shifting attention to more audience involvement upsets engrained journalistic routines and building new ones requires time, effort and organizational buy-in, which some newsrooms are struggling to achieve.

For academic research, audience engagement is a moving target with ever changing boundaries. That doesn’t necessarily make prior research, such as Nelson’s, outdated. It just means that these findings need to be understood as a step along the way. Part of the issue here is that audience engagement is not evolving in a vacuum but in an ecosystem of practitioners and entrepreneurs, funders and researchers. All of them are actively constructing and contributing to emerging norms, values and beliefs. A lack of shared standards, definitions or practices is not a symptom of weakness but a sign of institutional change that we yet have to fully grasp.

Thomas R. Schmidt is a postdoctoral fellow at the Agora Journalism Center, School of Journalism and Communication, University of Oregon.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store