Introducing: The Email Newsletter Benchmarking Tool

We built a free and open-source tool that allows newsrooms to measure and benchmark the email metrics that matter.

Hong Qu
Hong Qu
Nov 13, 2018 · 6 min read

Why Email Metrics Matter for Newsrooms

We know that email newsletters are a vital vehicle for audience development, and that’s why we are conducting research — both qualitative and quantitative — to understand the dynamics of operating successful newsletters.


What Is the Email Benchmarking Tool?

We are building out the next incarnation of our original research in the form of the Email Benchmarking Tool as a more accessible service for deriving actionable insights from the analytics. Currently, the tool is available only to MailChimp users.

What Does It Do?

The Email Benchmarking Tool provides reports that can be used to improve the health, growth, and engagement of any newsroom’s email products.

The Benchmarking Element

For all of those metrics, with the exception of the distribution of your open rate, the tool gives you both your newsletter’s metric and the average metric for all other users of our tool (“average” referring to the mean across all lists in our data set).

Example of an output from the tool —metric #2 List Composition (see below).
  1. List Composition: This metric breaks down the total number of unique email addresses from the entire list into percentages of users who are Subscribed (current subscribers), Unsubscribed (subscribers who removed themselves from list or whom the list owner removed), Cleaned (subscribers removed from the list, typically by a service provider after email bounces), and Pending (semi-subscribers stuck in the limbo of double opt in — or, someone who gave their email address but did not hit the confirmation button in their email inbox). By looking at this metric, you can see whether you have an Unsubscribed or Pending problem that needs to be addressed.
  2. Overall Open Rate: This metric, another “vanity” one, shows your list open rate. While list and campaign open rates are the traditional ways of looking at your email performance, a better alternative for examining your list open rate is through a distribution of your subscribers’ individual unique open rates (see next).
  3. Distribution of List Open Rate: This metric shows the distribution of user unique open rates for current subscribers on your list. For example, you can see what percentage of your list opens zero percent, 10 percent, 20 percent, all the way up to 80 percent of the time. We think newsrooms should focus on retaining, growing, and monetizing that segment of 80 percent-plus openers; on moving users with lower levels of engagement along the engagement funnel; and on removing the users who stay inactive over time.
  4. Percentage Open > 80 Percent: This metric shows your most engaged subscribers — those who open between 80 percent and 100 percent of your emails. These subscribers deserve greater scrutiny: what drives their activity? What are a few user personas that capture this audience? It is worth trying to figure out how to move more of your subscribers into this portion of your list.
  5. Percentage Inactive in Past Year: This metric shows you the percentage of current subscribers on your list who haven’t opened one of your emails in the past 365 days. Inactive subscribers can cloud your metrics, affect your email’s deliverability, and make it harder to understand your list dynamics. We recommend that newsrooms manually remove those inactive users from their list (after sending them a re-engagement campaign).


The Single Subject News Project

A research project of the Shorenstein Center on Media, Politics and Public Policy at the Harvard Kennedy School. We study how nonprofit, single subject news sites can engage, grow and monetize their online audiences.

Hong Qu

Written by

Hong Qu

digital tool maker, program director for technology at Shorenstein Center

The Single Subject News Project

A research project of the Shorenstein Center on Media, Politics and Public Policy at the Harvard Kennedy School. We study how nonprofit, single subject news sites can engage, grow and monetize their online audiences.