This Is Fine

While it’s not new news, many traditional digital publishers are in trouble. For years, they’ve ceded more and more control to the big players in the space, and there may be no looking back. A handful of giant, early internet unicorns that have large stakes in ad tech (Google, Facebook, and to a lesser extent Amazon and Twitter) have been handed the keys to content distribution, content presentation, monetization strategy, and now even paywall management, leaving many pubs feeling uneasy and helpless.

In an effort to quantify the problem, I looked under the covers at the network traffic of ten popular US news sites. I chose news because much of it is commoditized (yeah, yeah, there are exceptions — more on that below), and publishers who rely on heavily commoditized content are among the most at risk.

tl;dr: A lot of you guys are screwed.

Methods

Before I go into the analysis, some notes on my methods:

• I examined the HTTP traffic that loaded on ten top news sites using Chrome’s ‘Inspect’ tool.
• I cleared the cache (but not cookies) before each test.
• I categorized each call into ‘Content’ (the website’s own content), ‘Ad delivery related’ (anything related to ad serving, bidding, targeting, or content recommendation [like Outbrain]), ‘Measurement’ (mainly viewability and verification), and ‘User syncing’ (calls out to third parties to pass and sync user IDs).
• If I wasn’t confident in what a third-party call did, I put it in ‘Content’, so the ad-related calls are likely underreported.
• For each page visited, I waited for the viewable area to load, and then slowly scrolled to bottom of the page. For sites with endless scrolls, I tried to scroll down enough to see 4–5 ads.
• Note this analysis was for desktop only. I’ll try to follow up with mobile data in the future.

Analysis

Let’s dig in. I observed over 6,000 HTTP requests across the ten sites. Here’s what they did:

Only 49% of calls were related to delivering actual content to the user. On these sites, the ads take up relatively little real estate relative to content, so this stat shows just how much work goes on behind the scenes to serve them. Some sites have it worse than others:

What’s surprising here is that even some publishers known to focus acutely on user experience still have a lot of calls going out to ad companies. Look at the Washington Post with nearly 60% of calls related to monetization — all to display only 4 ads (and 3 of them were house ads)!

Across all ten sites, I observed 51 ads. How many calls does it take to serve an ad on average?

OMG, Daily Mail, 94 requests? Even the best on the list, boston.com, still requires 41. (And yes, some of these requests might not be directly ad-related, but the vast majority of them are.)

What’s Up with All Those Measurement and User Syncing Requests?

21% of all the calls I analyzed were for user syncing or measurement. This is new and remarkable. It also shows why the market values companies like Moat, IAS, and Acxiom/LiveRamp so highly. But why so many calls?

For measurement and brand safety, it’s pretty clear. We’ve been dogged by a trust and transparency problem for more than a decade, and brands finally got wise and wanted proof that they were getting what they paid for. The big guys — Facebook, Google, even Twitter — do not have this problem. They tightly control their entire stack: the display, the formatting, security, the number of third parties that have access, and the content. This translates to advertiser-friendly things within their gardens like inherent safety and predictable viewability. Dividing the number of measurement/IVT calls by the number of ads per site, we get:

That’s a lot of measurement per ad. Why don’t we employ sampling here? In a campaign that’s serving millions of impressions, do we really need to measure every one? Google doesn’t sample, and the MRC isn’t always clear on whether it’s allowed, though they seem to have a preference when it comes to IVT:

While the IVT Guidelines do not expressly disallow sampling of IVT measurement, such approaches are not in accordance with the impression-level premise, which is stated as the preferred granularity of measurement. — Media Rating Council, 2017

Facebook, Google, Twitter, and News 
I also looked at twitter.com, news.google.com, and facebook.com — all popular sources of news. On Twitter and Facebook, I scrolled down far enough to see 4 promoted ads. None of these sites allowed any third-party calls (excepting a few to Google Analytics on Twitter). They were also among the few sites I looked at that had 100% HTTPS traffic. And on average, they made about half as many requests out of the browser than the ten news sites I looked at. Why? Because they don’t call dozens of third parties to help them monetize.

For user syncing/cookie matching, it’s also pretty clear. Nearly ten percent of all the traffic I observed was related to user syncing. Wait…what?

With so much audience going to the big guys (according to some sources, two-thirds of Americans get at least some of their news from social media), independents are pressured to work together to construct a patchwork identity graph across hundreds of companies. The advantage goes to the walled gardens that don’t have to worry about messy cookie syncing, inconvenient match rates, and privacy issues over sharing user data with other parties. And the penalty is called on consumers, who have to deal with this ad tech tithe that don’t benefit them in the least.

Here’s the data, the average number of syncing calls per ad:

Consequences

These requests flying around to n number of parties impose severe penalties on pubs. If you’ve ever worked at a publisher (especially in ad ops), you know that shit breaks all the time because of this complexity. Again, the big guys are more or less immune to these issues because they maintain such tight control, first-party serve most everything, and have a first-party relationship with their users (i.e., they know a lot about you already), but the end result for pubs is ads that break frequently. Here are just a few of the examples I saw on the sites I analyzed:

What Can Publishers Do?

One way to fight back is to join coalitions like DCN’s TrustX, basically fully trusted marketplaces, but they haven’t succeeded in the past. Anyone remember the Newspaper Consortium? Other efforts like TAG and ads.txt should help, but only if publishers get on board. Antitrust laws and ubiquitous paranoia around of pooling of resources and assets with competitors make such effort difficult.

Paywalls to the rescue? Maybe. While there’s no shortage of ways to get around them, only a few companies are doing this well. The New York Times managed to boost subscription revenue by 40% Y-O-Y last quarter on the strength of its strong brand, clutter-free presentation of content, and commitment to focus on subscription business over advertising (though digital ad revenue was still up 19%). It aggressively tests various tactics, and take steps to ensure its inventory is harder to access for free, putting extra content like its popular cooking site and crossword app behind paywalls.

Facebook (and more recently, Google) announced that it’s testing a paywall product to help media companies sign up subscribers, and while the company says it doesn’t want a piece of that revenue, is it smart for publishers to give up collections to the aggregators along with everything else? Pubs who can should do this themselves, or look at companies 100% focused on helping them.

Browsers to the rescue? Nope. While it doesn’t look like our fears around ad blocking in the mobile space will be realized, it’s still an issue on desktop. The IAB says 26% of internet users have one installed. That’s a lot of lost revenue, and Google has taken notice, announcing that it will include ad blocking functionality in future version of Chrome. Sure, this will be great for consumers, but it will be equally great for Google, which owns half of the browsing market with Chrome. And it will leave many publishers that have been experimenting with different ad types and formats scrambling to become compliant with whatever Chrome’s new rules are.

Differentiated content and presentation is a bright spot, but only if you have it. Publishers without it will have a hard time convincing users to pay for information they can find in a hundred other places. I can read about Trump’s latest tantrum anywhere, so how can news sites stand out? Ben Thompson’s wonderful article ‘Publishers and the Pursuit of the Past’ suggests that it’s the “production, unlike the articles themselves, [that] can be differentiated and sold as a scarce product.” Will publishers go the way of so many brick-and-mortar stores? How many publishers will morph solely into content producers without a means to display their own content? Can we learn by taking lessons from direct-to-consumer brands like Warby Parker, Casper, and Dollar Shave? My bet is on the sites that play to brand strength, editorial integrity, and top-notch presentation over the common denominator of click-baity ‘content’ recommendation tiles, obnoxious ad formats, and dubious data collection practices.

I asked a friend for feedback on this article, a programmatic expert, and he reminded me that publishers have choices. “If you don’t want 15 SSP calls,” he wrote, “don’t work with 15 SSPs.” While 100% true, I don’t think it’s that easy. Publishers aren’t hard-coding all of the 6,000 requests I observed onto their sites. Their ad partners and vendors are doing that for them. One SSP tag can spawn dozens of tags to parties that a publisher has no relationship with. Even if the returns from working with a new partner are slightly incremental, they are still incremental, and once you open that door, it’s hard to shut it.

Don’t get me wrong. Google and Facebook are great companies, providing valuable utility, and clearly most consumers agree. Nonetheless, publishers have been disrupted by new distribution models controlled by these guys, and they went along for the ride. Their tendency to aggressively chase short-term revenue — and all the shitty ad formats, pixel bombing, increased ad blocking, and general lack of control that comes along with it — will haunt them in the future as users flee to consume content in friendlier arenas controlled by companies that already see nearly every person in the U.S. every single day. As a favorite author, David Mitchell, wrote, “if you bare your arse to a vengeful unicorn, the number of possible outcomes dwindles to one”.