Using evaluation to help public radio achieve its strategic goals

I recently attended the American Evaluation Association’s annual conference. At one of the workshops, the facilitator began with one of those interactive polls in which everybody’s input is projected live before the group. The question was simple: What is your professional area of focus? This being a group of evaluators, things like “Health care” and “Education” grew larger and larger in the dynamic word cloud, and tucked away somewhere and getting smaller by the second was my “media,” my response.

Within the field of evaluation, media assessment is a niche, but it has the potential to do for media and journalism what evaluation does for other fields: figure out what we’re doing well, whether we’re doing “good” in the world, and if we’re achieving our goals.

For the past four years, CapRadio has worked to make community engagement a core element of the station’s multimedia documentary series called The View From Here. This series is community-centered, emphasizes face-to-face engagement with the audience, and invests in long-term reporting and community engagement that require a lot of staffing and resources. Based on TVFH’s success, CapRadio has developed a new strategic objective to translate the model TVFH pioneered and integrate community engagement in to regular reporting processes.

In 2018 alone, The Impact Architects has been part of two evaluations for Capital Public Radio, the NPR affiliate in Sacramento, California. I led the most recent project, evaluating CapRadio’s Rural Suicide community engagement and reporting initiative, which was funded by USC’s Annenberg Center for Health Journalism. The project shows the value evaluation can offer to a media organization and is a proof of concept that evaluators can be key collaborators in helping newsrooms meet their strategic goal.

Taking one successful strategy and using it in another similar but different area makes sense. But the question remains: how to do that? One of the recommendations from our earlier June evaluation suggested a method: “Develop and prioritize standard community engagement practices such that CapRadio can select from a ‘menu’ of types of pre-reporting engagement, community events, and content, rather than develop custom initiatives for each project.”

CapRadio took the next logical step and began to test replicable engagement models, and we focused on testing one model through our recent evaluation of the station’s Rural Suicides initiative. Developing and identifying actions for the “menu” fit CapRadio’s evaluation priorities and would help us move toward a clearer picture of what community-engaged journalism could look like at scale.

Through the interviews I conducted with community partners, people in the newsroom, and jesikah maria ross, the CapRadio staffer who has spearheaded the station’s engagement work, including TVFH and the Rural Suicide project, I heard about a lot of practices that could start a list of options for future community engagement. The conversations with the stakeholders showed why people thought practices were valuable, as well as what they could do next time.

I organized the different practices based on resources required and the scope of work:

• Things anybody can do, right now, with limited resources. Asking modified questions; “what do you want to tell me about this topic?” instead of “I have some questions about this topic I’d like you to answer.”
• Flexible options for anything from a single story to a multi-part series. Callouts for stories related to a topic using either an online tool or a Google form, possibly with the help of a media collaborator to target the call.
• Options appropriate for multi-part series that require resources. Identifying well-networked community partners already invested in the topic.
• Resource intensive options appropriate for multi-part series or long-term projects. Facilitating community events about a topic.

One example of a practice was collaboration with local newsrooms. CapRadio reached out to the local radio station, KVGC, and paper, Amador Ledger Dispatch. Doing so amplified CapRadio’s stories to reach a potentially new audience. This example fits in the “Flexible options” category because it mostly required making contact with the media outlets to introduce the series and discuss its relevance for their audience. The ask, however, was ultimately about airtime and column space. There’s room to do more, as Olivia Henry from the Annenberg Center for Health Journalism told me. (Henry, notably, was an additional engagement resource that helped drive the project along.) With enough forethought and planning, relationships with local media can be strengthened, which would make true collaboration possible to get an even greater and more positive response from communities. The idea is there for next time.

It’s clear that there is no one-size fits all form of community engaged journalism. And there’s always a question of resources, support, and, as The Agora Journalism Center recently found, readiness. What makes sense for one project may not make sense for another. Evaluation’s value-add for CapRadio has been incrementally building in an advancement of its strategic goals. And, similarly, the value-add for this niche of evaluation professionals who work in media is that this work, too, can build toward something greater. The “menu of options” idea is eminently transportable, and it is appealing precisely because there’s no one-size fits all approach. Indeed, it’s one of the major questions The Impact Architects is thinking about on a larger scale right now.