How we think about the impact we want to have at WhereBy.Us

Journalists have been talking about how to measure the impact of their work for a while now. In his 2013 white paper, ProPublica’s Dick Tofel defines impact as “the change and reform that journalism has spurred.” Lindsay Green-Barber, former impact analyst for Reveal who helped build an open-source Impact Tracker tool, defined impact as “all the real-world change associated with your work.” And Chalkbeat’s Elizabeth Green, Philissa Cramer (and me) wrote this 2014 whitepaper that defined journalistic impact as evidence that our stories have influenced either informed action, the actions that readers take based on our reporting, or civic deliberation, the conversations readers have based on our reporting.

That’s all pretty heady stuff. And while there’s value in tracking this type of journalistic impact, at WhereBy.Us we think there’s also value in broadening that definition of impact and capturing other (seemingly “smaller”) ways we’re adding value and usefulness to our readers’ lives. In other words, we think there’s value in consistently tracking whether we know what we’re doing matters to our users.

Our WhereBy.Us tagline is “live like you live here,” and so we define impact as all the ways we’re helping our readers do that. In each of our five cities, two-person editorial teams are focused on helping people feel more connected to where they live. We don’t focus our journalism on attending every City Hall meeting or requesting public records for deep investigations. We provide a different type of essential public service –– helping our readers make sense of what’s going on in their local communities and showing them how they can get involved.

That means we needed to build an impact taxonomy that worked for what we were making, for the impact we want to have.

So here’s a step-by-step process for how we created our impact taxonomy and a simple impact tracker tool, and how we’re planning to use it to collect data that we hypothesize will help us drive revenue and build a sustainable business.

Step 1: We came up with a very rough definition of what impact means for us.

Many news organizations aim to serve different audiences, so it makes sense that our definitions of the kind of impact we want to have will vary. But there are a few guiding questions I think every newsroom can use to figure out how they want to define impact:

• If you’re talking to a source who’s never heard of your publication before, what stories do you tell them you’ve written to earn their trust and to build credibility with them?
• If you’re catching up with a friend you haven’t talked to in a while and you want to tell them what you’ve been up to at work, what do you share with them first?
• When you’re applying for a job and are asked for clips, which ones do you include?

Once you think about your best work, you should be able to see a common thread, and that should inform the basis of the kind of impact you want to have.

Step 2: We identified all the places in which we receive feedback from our stakeholders.

We made a big list of all the places we currently hear from our readers, sponsors or anyone else who has read or experienced what we make. That list included things like reader emails, our social platforms, in-person feedback moments, user research and reader feedback from our Net Promoter Score surveys. Making this list helped us understand the scope of the data we were trying to collect. Later, we’d think about the best ways to collect data from each of these places so we aren’t just relying on our memories when it comes time to rounding up all these impacts. (If you didn’t already know, we’re big fans of user research and making giant lists to better understand our work.)

Step 3: We looked at that list of sources and, for each one, pulled what we thought were relevant “impacts” for the past three months.

A group of us spent about an hour and a half looking through these impacts. We each took one source and added to a spreadsheet what we thought the relevant impacts were. This is boring, manual work, but I think it’s a step that’s often missed and can lead to an impact taxonomy that doesn’t account for the kind of impact you’re having right now. Of course it’s important to be ambitious (and maybe one day WhereBy.Us will add something like “policy change” to its taxonomy) but if you make a taxonomy that’s overly ambitious and you aren’t able to input impacts you’re having right now, no one will ever use the thing.

Step 4: We created an impact taxonomy based on similar themes we saw in the impacts we collected.

After we added some impacts to the spreadsheet, we read through the data everyone collected. Then we took a stab at creating categories for the common themes we saw. That led us to create this impact taxonomy:

–– Informed awareness: We helped build awareness.

• Influencer feedback: We provided local leaders and influential organizations useful content they wanted to share with other people.
• Media mention: Other media shared or republished something we made.

— Informed action: We helped our users take action.

• Reader action: Our users took action based on something we made or shared.
• Reader contribution: Our users helped inform or contributed to our editorial process.

— Belonging/connection: We helped our users feel more connected to the city and to each other.

• Reader belonging: We helped fuel our users’ belonging in their city.
• Reader connection: We helped our users get connected to people they wouldn’t have met otherwise.

–– User Love: Based on our users’ feedback, we’re achieving our company mission.

• Sponsor/partner feedback
• Events feedback
• Storytelling feedback
• Member feedback

— External recognition: We were recognized by our industry or our community with these awards and honors.

• Award: When we’ve been recognized with an industry award or some other type of award.
• Media recognition: When our team has been interviewed by other media, made any public appearances on behalf of the company, or our company has been mentioned by other media.

Step 5: We used Airtable to collect and organize existing and future impacts.

As previously mentioned, there are other tools out there that are much more sophisticated — like CIR’s open-source Impact Tracker tool and Chalkbeat’s open-source MORI tool. But we decided to build an MVP and then decide later whether it’s worth making a deeper investment.

We added a tab to our existing editorial calendar in Airtable so that we could link the impacts to stories or newsletter items we published.

It looks like this:

Besides adding an impact directly to the table, we also created an Airtable form and an /impact Slackbot to make it as easy as possible for anyone in the company to add impacts.

We also thought about how to divide up the impact tracking based on who’s closest to the feedback source. For example, we have a customer support specialist who reads all the responses that come through our Net Promoter Score surveys. So it’s now his responsibility to choose which responses feel like they meet our criteria for impact and add those to our impact tracker.

Step 6: We trained the team.

As part of creating training materials and documentation, we also added a handful of impacts each city has had so far so there were some examples to help inspire the team.

And to help everyone understand what all this was for, we emphasized our hypothesis: “Tracking our impact can help us make more money and become sustainable. We can use impacts to show existing and potential sponsors, members, crowdfunders, and investors how their dollars would be used to directly make an impact in communities they care about.”

Which leads us to why…

Step 7: We created an impact report.

A few weeks after introducing the impact tracker and giving everyone time to add their impacts from the last two quarters, I sifted through the impacts in the tracker and pulled the ones that felt most relevant. I grouped them based on themes and wrote up a short summary of the impact each city has had recently. For future iterations, I’ve suggested to the team to create a template based on our taxonomy. I shared that impact report draft with our teams who are editing/reviewing it now.

My hope is our team’s day to day work will inform future iterations of the taxonomy and impact collection. But for now, it feels pretty awesome to have all this stuff in one place and in a format that you could give your mom to read!

How do you define the impact of your work, and how do you track it? We’d love to hear from you.

Anika Anand is the director of product at WhereBy.Us and cofounder of The Evergrey. This post is part of a series on how WhereBy.Us works in an effort to share what we’re learning, and to learn from others across the industry. Thanks to WhereBy.Us COO Rebekah Monson and growth editor Alexandra Smith for editing.

Interested in WhereBy.Us coming to your city?
Tell us where we should head next.

Sign up for our WhereBy.Us city newsletters!
The New Tropic
The Evergrey
Bridgeliner
Pulptown
The Incline