Ask And You Shall Receive: The Power of Crowdsourcing For Public Radio

This article is co-authored with Tonya Mosley, Senior Education Reporter, WBUR 90.9 FM, Boston.

When Tonya Mosley arrived at WBUR 90.9 FM (Boston) last summer, she was tasked with covering the debate around the expansion of charter schools in Massachusetts. Election fervour was rising to its final peak, and Question 2, a measure to ‘lift the cap’ on the state’s charter schools was one of the more contentious questions on the ballot.

The pro-charter and pro-public school camps were sharpening their arguments and framing their narratives through an onslaught of television and radio ads. Volunteers from both camps were going door to door in an attempt to inform the public on their respective arguments.

As experienced journalists know well, there is often a history to long-running controversial public policy. Massachusetts and greater Boston are no exceptions. As local newsrooms will attest, there is also local history and real, lived, and diverse experiences that parents and teachers have on matters like charter schools. Those experiences sometimes fall somewhere in between the “for” or “against” categories.

How would WBUR report on Question 2 in a way that gives depth, context and useful information? The conventional approach would be to get out there, meet people, hopefully identify new and diverse voices, and air a bunch of great stories on the issue.

What we did

What we actually did was different. Earlier this year Subramaniam (Subbu) Vincent experimented with a crowdsourcing technique at KALW 91.7 FM (San Francisco). Crowdsourcing he learned, could surface detailed insights, practical nuances and experiences from the community on specific local issues ahead of reporting. At KALW, Vincent experimented with Screendoor (an advanced form tool) to engage with KALW listeners. Vincent, along with Mosley and her editor Louise Kennedy, decided to give this same approach a try at WBUR. The team asked both listeners and the online audience not only their thoughts on Question 2, but their experiences with both traditional public and charter schools in Massachusetts.

A quick note about Screendoor: This is a form tool that lets the user create complex, dynamic forms that evolve into what’s called ‘conditional branching.’

Here’s how this works: If a respondent answers ‘yes’ to a question, they are then shown a ‘next’ question that is different than if they had answered “no.” There are different set of questions for teachers, parents, students and concerned citizens. Responses go into the Screendoor tool on the web, which users login and view. There is no software to download. A fairly complex questionnaire targeting multiple types of citizens will still only take a minute or two, because all the respondents are not seeing all the questions. Also, no major tech is needed at the newsrooms since it runs off the cloud, or the software-as-a-service model.

Mosley and Kennedy planned out the execution of the crowdsourcing survey with Vincent over one video call and a few email swaps. Vincent had developed a simple execution checklist on how to start from zero and go to launch with the callout. The three first discussed the specific archetypes of people to target for the callout. The topic is as vast as it is heated. There are parents, teachers, educators, academics and public policy scholars, bloggers and semi-think tank folks, administrators, consultants and so forth. A wide range of these groups are familiar with WBUR.

We decided to keep the focus on parents and teachers, since these two groups are at the core of the issue; and we framed the questions to them. We included questions first about their school experiences, before moving to the ballot measure.

It took about two weeks to prepare the questions and the flow of the survey. Vincent then showed Kennedy how to create the conditional-branching form. Vincent was in Northern California, and Kennedy in Boston. In the third week we embedded the form in the Help Us Report On The Charter Schools article.

Here’s how the article opened:

The callout article was headlined with a ‘Help us…’

The callout-form was embedded at the end. And here it how it looked to a first-time visitor. Note: There is deliberately no indication of how deep the form might or might not go.

The screenshots below give you a glimpse of how the form changes depending what you might have clicked as a citizen respondent. The key point is that it is setup for most respondents seeing only the parts that are relevant to how they are proceeding.

We setup some email customizations in the form for personalized responses. We reviewed our plans with the social media anchor so that they would get some mileage for the callout article on WBUR’s Facebook and Twitter channels. We created accounts for Mosley, Kennedy and Vincent, using the 50$/month plan from Screendoor. This allows five users and five active callouts.

We went live August 15th, with a response deadline of September 15th. The topic was a heated one, and by the end of the first day, even before the social media posts were published we received responses. By the callout deadline, we had received 335 responses, around a 100 each week, with a slowdown in the fourth week. We tagged each response manually with labels that made sense to us. It helped us count the number of responses across different types.

Many responses were completely unexpected, authentic and original. Other responses validated our underlying sense of the issues that were at the heart of the debate. A few responses also alerted us to upcoming events we may have otherwise missed.

Screenshot: The searchable database of responses to WBUR’s callout, in Screendoor (cloud login), seen by editors and reporters. Citizen responder names greyed out to protect identities.

Mosley found respondents with complicated backstories the most compelling. For instance, one survey respondent worked in a charter school, had one child in private school and another in public school. The respondents perspective shed light on the complexities behind many parents opinions about charter school expansion. We marked these types of responses for follow-ups. Under each response is a space where reporters and editors can write notes.

Between September 15th and until November 8th, WBUR produced four stories that leveraged the callout engagement. They are cataloged here: http://www.wbur.org/edify/tag/crowdsourced.

One story in particular, focused on the impact of a charter school expansion on the suburbs outside of Boston. Mosley followed up with a respondent who lived in the suburban town of Lexington. The respondent was passionate and had unique life experiences that added richness to the arguments over charter schools. Mosley believes the structure of the crowdsourcing survey gave the respondent a vehicle by which to share insights she might have not revealed otherwise.

In each story that used an interview with a respondent or insights from the callout, the article online acknowledged the callout after the byline. The week after the elections, Mosley and Kennedy’s education team “Edify” sent out a thank you note from the form tool to all the respondents with a link to the stories.

And so what?

Simply put, the callout was a success. We learned a lot. The benefits applied to both Mosley, who was new to Boston and her editor Louise Kennedy. The costs were the one-time effort of two video calls, a few email swaps, and the little bit of planning.

For the reporter

  • A good way to feel the pulse and get a deeper sense of an issue or topic.
  • Useful to get sources for reporting and background.
  • Especially useful for reporters new to a city searching for authentic voices.
  • Discovery of nuances useful in adding richness to stories.

For the editor

  • Gives room for open thinking on the question “What should an editor be doing?”
  • Time well-spent in brainstorming questions with the reporter(s) and designing the form.
  • Experienced editors and longtime city residents might find validation of collective wisdom in the responses. Editors are asking “Is this the right story?” and callout responses help there.
  • Discovery of new nuances is useful for editors too.

Institutional benefits

  • The database of sources (names, email addresses) as well the collected insights/responses are archived permanently and digitally. It is searchable through keywords.
  • Six months or a year after a project and stories have been done, a new reporter can access the rich repository to educate himself or herself before a new plan is done. It can bring people up to speed in ways that go beyond reading stories published.
  • Documents submitted via this system are also accessible.

The sources can be exported into any other CRM or source management tool if the newsroom already uses one.

What next?

Crowdsourcing is an amazing tool that allows journalists the opportunity to get even closer to the truth than ever before. It acts as a pre-reporting guide, allowing the reporter to tap into communities he/she perhaps may not always have access to.

Mosley plans to use the crowdsourcing tool regularly to add richness to her stories and build a database of sources. The biggest challenge will be getting our crowdsourcing surveys to as many communities as possible. One way to do this is to partner with organizations that have direct connections with communities public radio may not regularly interact with. We are working through this idea now.

Mosley will also experiment with ways to report out crowdsourcing responses. For instance, some responses offered great opportunities to tell the stories through visuals (photos & video) — some offered anecdotes which are great for social media — and others were great characters that offered strong narratives for radio.

Update: Around the time this article was published, WBUR’s latest callout went live. It is asking the crowd to weigh in on math learning and children.

Coming next: An executable, do-it-yourself guide to crowdsourcing insights for your own local newsroom.

Subramaniam Vincent and Tonya Mosley

Tonya Mosley and Subramaniam Vincent were 2016 John S. Knight Fellows at Stanford University. Mosley is currently the Senior Education reporter at WBUR in Boston. Vincent is prototyping a civic radar screen solution for local newsrooms and citizens in US cities, starting with San Francisco. He was previously CEO and editor-in-chief of a startup civic media venture, Citizen Matters, in Bangalore, India.

Acknowledgments: Thanks to Amanda Zamora and Terry Parris Jr. for sharing their experiences on the use of Screendoor at ProPublica, via the Crowd Powered News Network. And to Ashley Alvarado, Annie Anderson, Daren Brabham, Jennifer Brandel, and Andrew Haeg for various conversations.