For $1 a month, you can supplant digital ads. Publishers, are you listening?

We’ve all seen our share of pop-up ads. They take over the page, block our view of whatever we had come online to see. We hunt and peck around for the little ‘X’ that will close them. Most of us do, anyway.

It’s easy to say that they’re annoying, but how annoying are they? And how much would it cost a news outlet to stop annoying its readers in this way? Is it possible removing pop-ups could encourage more people to become paying subscribers?

The Bay Area News Group, publisher of the San Jose Mercury News and East Bay Times, is finding out.

The question is sensitive because pop-up ads are very lucrative. They can make $45 per 1,000 impressions — about what advertisers pay to be on TV. But they are increasingly going to be a relic of our digital lives. Search giant Google has declared it will start flagging sites that have pop-ups and other distracting ad formats, and shut off all ads directed to the offending site from its Chrome browser starting in February if the site hasn’t removed such ads for 30 days. As a reminder, Chrome has more than a 50 percent share worldwide, and this move would be devastating to any publisher that runs afoul of these rules.

But for the benefit of research, we decided to decorate one of these ads with all kinds of custom code that will enable the tracking of users that interact with it.

It was my goal as a John S. Knight Journalism Fellow at Stanford last year to see whether news junkies would pay to have ads removed from their online experience, much like video and audio subscribers pay extra to remove ads on Hulu, YouTube and Pandora. Ad-free has become a thing in digital entertainment. The aim is to find out whether it applies to news sites as well, so that subscriptions can sustain journalism in a way that moves the industry away from clickbait and fake news, and more towards providing a valuable service that users will pay for.

As a recipient of a grant from the Jim Bettinger News Innovation Fund for JSK alumni, and with the backing of my employer, The Associated Press, I am continuing to experiment with the idea out in the real world.

And thanks to the good graces of Executive Editor Neil Chase and Managing Editor Randall Keith, I’ve been sitting in on a weekly meeting at BANG’s headquarters in San Jose as they troubleshoot the re-launch of their digital paywall on multiple sites. So far, that part of the transition to a digital future is working. Many new people every day fish out a credit card from their wallets to subscribe, once their supply of free article views on or runs out.

Neil and his team have graciously opened up their vaults of data and tracking tools to me. Together with ad ops data and platforms director Phil Lee, coding expert Matti Lehto at ResponsiveAds Inc., ad designer Arnette Leonetti, and AP research associate Anne Cai, there is a veritable skunkworks team tackling the question of ad sensitivity at BANG.

The ad we commandeered was scheduled to run in the last two weeks of December up to 40,000 times on article pages in the “News” section of Unique visitors would only encounter it once every 12 hours. After we tagged people who had interacted with the ad, we looked at their activity on the rest of the site.

Here’s what we found.

What did people do when they saw a pop-up ad on a news article at 58.9% closed it; 29.5% saw it and took no action; and 11.6% clicked through to find out more. Data was collected between Dec. 16, 2017 and Dec. 29, 2017.
  1. Most people closed the ad.

Nearly 59 percent clicked the ‘X’ or clicked outside the ad’s borders to close it immediately, a clear sign that they weren’t interested in the offer or were bothered by the intrusion on their way to content. In any case, they’d seen enough.

(We won’t name the advertiser or what it was selling here, other than to say it was related to New Year’s Eve. We’re well aware that this could be driving some of the actions, but we’re going to test other content and advertisers as well in the coming days and weeks.)

Nearly 30 percent saw the ad and did nothing, meaning they let it close automatically after 8 seconds or otherwise ended their session and we lost sight of them.

Nearly 12 percent clicked through to learn more. If you are stunned by that click-through rate, you’re not alone, as industry-wide click-through rates can be fractions of a single percent depending on the format. Independently of our custom code, regular ad-tracking also showed the ad had an amazing click-through rate of 8.72 percent, which is more than 7 times as good as the next most-successful pop-up ad on BANG properties in the past year.

Let me add: The ad took over most of the screen on desktop and mobile browsers and more than 60 percent of the impressions were delivered on mobile. There could be some fat-finger effects as people tried to close the ad but accidentally clicked to open it instead. That, or people were somehow desperate for something to do on New Year’s Eve.

How engaged are people, depending on their behavior? During the data collection period from Dec. 16 through Dec. 29, 2017, of the people who saw the ad, the people who clicked anywhere were generally more engaged. Those who clicked through to find out more saw 17.05 pages on average during the period; people who closed the ad saw 16.69 pages; and people who saw the ad but did nothing saw 9.4 pages on average. (The Total Saw Ad group combines these three groups together, which themselves are mutually exclusive). Those who didn’t see the ad saw 3.14 pages on average, while people who clicked on a “subscribe” button for full access saw the highest number, 24.81 pages on average. (There is some overlap between those who clicked “subscribe” and the other groups)

2. The clickers were more engaged.

Whether people clicked to close the ad or clicked to see more, these people generally consumed more pages than people who did nothing. Of those who saw the ad, those who clicked through saw 17.05 pages on average; those who closed the ad saw 16.69 pages on average; while those who saw the ad but did nothing saw 9.4 pages on average. As a group, all of those exposed to the ad saw an average of 14.58 pages, much higher than the 3.14 pages of people who didn’t see the ad.

(Almost certainly, the vast difference in pageviews between those who saw the ad and those who didn’t is mainly due to where we showed the ad: on “News” section articles. This in itself tells us that people who read news stories are more engaged than those who read other kinds of stories. Again, we plan to test ads that run in different sections or all of them to sort out this effect.)

Meanwhile, the test confirmed that those who clicked “subscribe” are the most active users and clearly get the most value out of the site. (Whether they actually paid up or not is another question, but they did express an interest in a subscription offer). They saw an average of 24.81 pages over the period.

How much ad revenue did people generate, depending on their behavior? Those who clicked “subscribe” on a paywall offer generated the most estimated ad revenue over a 30-day period, at $1.11; those who clicked through on the ad generated an estimated $0.76; those who clicked to close the ad generated an estimated $0.75; those who saw the ad but did nothing generated an estimated $0.42; while those who didn’t see the ad (and probably didn’t see any “News” section stories), generated an estimated $0.14.

3. The cost of supplanting ads is $1.11 or less per month per person.

Now here is where the math gets a little tricky but bear with me. BANG uses both Google Analytics and DoubleClick for Publishers, also a Google tool. The back ends are connected, but not entirely. This duo of tools keeps track of ads that are automatically sent to BANG properties from programmatic ad exchanges, and some, but not all, of the ads that BANG sells directly. Combining these and boiling it all down to a pageview comparison, BANG quite consistently made 1.4 cents in ad revenue per pageview over the period. However, ad ops director Phil has to true-up these numbers every month or so, and consistently finds that this low-balls the actual ad revenue BANG brings in. By his estimate, one would have to increase the number by as much as 40% to get the real number. So to be conservative, I added 40% to every pageview value estimate, for 1.96 cents per pageview. Then I looked at the 13 days we measured and extrapolated that out to 30 days to determine how much in ad revenue each group is worth every 30 days. For those who clicked “subscribe,” the most engaged group, it’s $1.11; for people that clicked through on the ad, it’s $0.76; for people that closed the ad, it’s $0.75; for people that saw the ad and did nothing, it’s $0.42; and for people who didn’t see the ad (and probably don’t read “News” stories), it’s $0.14.

Could publishers offer ad-free for $1 a month? Maybe, but that doesn’t include the cost of all the engineering you’d have to put into it. And heck, people might be willing to pay even more than that. But, we’re talking about less than a single-copy price for most newspapers, and I’d venture that most “newspaper” publishers will tell you they won’t be in the business of printing ink on dead trees for much longer.

In the age of digital content, are people willing to pay for a cleaner news experience for such a modest price? I think so.

What is each group’s propensity to subscribe? During the test period from Dec. 16 through Dec. 29, 2017, 2.82% of people who clicked through on the ad also clicked “subscribe” on a subscription offer from the publisher; 1.49% of people who saw the ad and did nothing clicked “subscribe”; 0.93% of people who clicked to close the ad also clicked “subscribe”; and 0.05% of people who didn’t see the ad clicked “subscribe.”

4. People who click on the ad also clicked on subscription offers more often.

Let’s call them the curious: People who clicked through on ads — which themselves are offers — also clicked on the subscription offer of the publisher most often. The test showed 2.82% of those who clicked through on the ad also clicked “subscribe” at some point during the test period. I suppose that’s not too surprising. Perhaps they want to see what offers are all about. And we’re not (yet) tracking the online behavior of those who not only clicked “subscribe” but actually filled out all the forms and entered their credit card number. Like any online checkout, the drop-off between clicking “subscribe” and actually subscribing is significant.

I’ll admit I don’t have a good explanation for the next two groups. Why are people who saw the ad but didn’t close it more likely to have clicked “subscribe” (1.49%) than those who closed the ad (0.93%)? Could it be that people interested in subscribing are just naturally more tolerant of a heavy ad experience to begin with? It could be. Can we increase the propensity to click “subscribe” of the group who closed the ad by making their ad experience a bit less distracting? That remains something I’d like to look into further.

As for those who didn’t see the ad, I think we are still living with the baggage of not showing the ad to anyone who didn’t look at an article in the “News” section. My assumption is that people who read “News” are just more civic-minded, more interested in their local community, and more willing to see the value of a locally focused publication like the San Jose Mercury News. Meanwhile, because The Merc also has a global audience and its content is syndicated on many portal sites and social networks, a lot of people just visit once and are gone.

Add to all this one huge caveat: We did not track people who have ad blockers installed. By definition, they did not see our custom ad and are probably among the most ad-averse people out there. Yet they are lumped in with everyone who didn’t see the ad. We can say they generate $0 in ad revenue, but we don’t know how willing they are to subscribe.

To conclude, it’s still early days in our testing of ad sensitivity. I’ve got five more months of grant-supported time at BANG, and I intend to use them to explore this question as fully as possible. What further experiments should we pursue? Which ads bother people the most? How should the site treat people who come with ad blockers installed? (We’ll probably test this one.) What gets those people to subscribe? These are all questions I’d like to tackle. If you’d like us to pursue other ones or have any other thoughts, please drop me a line at In any case, I’ll keep you in the loop about what we find.

Ryan Nakashima is a grantee of the Jim Bettinger News Innovation Fund and was a JSK Fellow at Stanford in the class of 2016–2017. He is also a technology writer at The Associated Press based in San Francisco. He lives in Menlo Park, California with his wife and three kids.