Ads and data-driven web performance
Core Web Vitals and ads
In 2020 Google introduced the term Core Web Vitals which are
the subset of Web Vitals that apply to all web pages, should be measured by all site owners, and will be surfaced across all Google tools.
Along with this term, three metrics came into being:
- Largest Contentful Paint (LCP) — measures loading performance. It’s the time between when the page starts loading and the moment when the largest element in the viewport is rendered.
- First Input Delay (FID) — measures interactivity. It’s a delay between the moment when a user interacts with the page (for instance clicks a button) and the time point when user actions can be processed and handled by a browser.
- Cumulative Layout Shift (CLS) — measures visual stability. It used to be the sum of all layout shift scores that occurred during the entire lifespan of the web page and all data plots presented in this article are compliant with this definition. However, as of June 2021, the definition and way of computing this metric have changed (read more here) and currently, CLS is the maximum sum of individual layout shifts that occurred within the session window (time period when layout shifts happen with no break/short breaks between each other). The lower CLS the more visually stable is the web page and the target is a score of 0.1 or less. 75th percentile of page loads is considered to be a good threshold to measure the CLS metric.
Core Web Vitals are meant to help evaluate web performance and user experience on the web page.
I am a member of the Ad Engineering Team at Fandom which integrates the entire tech stack that is needed to display ads on our web pages. How does it connect with Core Web Vitals? Before ads display on the page, many things have to happen like some configuration steps, getting users’ consents in geos with data-privacy laws, and finally sending requests to Google Ad Manager which is our ad server. All these things lead to the situation when ads show up on the page after its content is loaded and this leads to page layout shifts. As you can imagine, user experience is not great when you try to read an article and it moves 100 pixels down because of an ad. It’s even worse when you want to click a link to another page but then, suddenly, it disappears and you click an ad instead. After an initial audit, it was clear that ads have a negative impact on the CLS metric on Fandom web pages and here our adventure begins.
Ads placeholders and their impact on CLS
The CLS metric was worse on mobile devices so we decided to start our work there. Back then, the Fandom mobile web view was generated by an Ember.js application that pulled data from MediaWiki API and then rendered page content. On Fandom mobile web pages you can find ad units located in different places across the page (see the image below).
We started with adding a placeholder for an ad slot called top leaderboard which was placed either at the top of the article or below an infobox (a set of information regarding a page’s subject presented in a table) if an infobox was present on a page. What is more, we aimed to avoid situations when the placeholder for an ad is much bigger than the actual ad and there is empty space around it, so we implemented placeholder resize (see the GIF below).
We ran our experiment with the top leaderboard placeholder on real traffic on one selected Fandom wiki and we were waiting for the performance data to be gathered. On the plot below you can see the CLS metric aggregated for 25th, 50th, and 75th percentile for mobile pageviews with top leaderboard ad impression, for the traffic where we ran our experiment. You can imagine a look of amazement on our faces when it turned out that in fact, we increased the CLS metric.
We decided to change the implementation and resize the placeholder on scroll — we thought that since it was a user action, CLS would not be computed after resize, and… we’ve made it even worse.
Eventually, we figured out what went wrong:
- First of all, page content was pulled from MediaWiki API and all mobile view elements, including the ad placeholder, were rendered on the frontend side. It caused a similar effect to the one we had with an ad without a placeholder — the placeholder showed up on the page after its content was rendered which caused layout shifts. However, with the placeholder, it was more likely that layout shift would happen before users scrolled to the place where it was placed (in case the ad was not in the viewport once the page loads).
- Second, according to the documentation scroll is not considered to be a user input thus shifts that happened within 500 ms are not excluded from CLS calculations.
Due to the first problem being a very big lift, we’ve decided to focus on the second problem first and we disabled placeholder resize on scroll. What is more, we changed the height of the top leaderboard ad placeholder. Initially, it was set to 250 px. According to the data analysis, top leaderboard ads with height less than 100 px were served on 70% of pageviews selected for the experiments so we decided to set it to 100 px in order to improve user experience.
Another experiment we carried out was adding placeholders for ads embedded into articles' content below the first viewport (we call them incontent ads). These ads are lazy-loaded and injecting placeholders for them was implemented in a way similar to lazy loading — when users scrolls down the page, consecutive placeholders were injected once ads’ requests were sent to our ad server. The below plots show how all these experiments impacted the CLS metric.
As you can see, the outcome of the experiments was not always what we expected, which is a CLS decrease. What is more, you can observe some CLS drops which were not related to our experiments. Taking into consideration that application releases happen on a daily basis, several dozen engineers are working on it and pages’ content is user generated, it can be challenging to evaluate which changes impacted the CLS metric.
More detailed analyses were done to differentiate the traffic and to find explanations for CLS fluctuations. However, in the meantime, Ember.js application which generated mobile views of Fandom pages started being replaced by a MediaWiki skin dedicated to mobile devices, and here is where our next adventure began. Knowing that the CLS metric was poor for this skin and that we can improve it by adding ads’ placeholders, we started working on it.
This time, we were able to render placeholders on the backend side and this is what we did for the top leaderboard ad.
Our team implemented another placeholder — this time for the top boxad ad which is an ad embedded into article content, below the top leaderboard ad. Although we were planning to implement it on the backend side, due to business reasons connected with ads performance, we decided to start with a frontend implementation and see how it performed. Before releasing our changes globally, we ran our experiment on selected traffic, and this time we observed the expected CLS drops. On the plots below you can see how these two experiments performed for given percentiles and in comparison to global traffic on skin dedicated to mobile views. We were rolling out these changes gradually and monitoring data and it took us weeks to release these changes to all of our global traffic.
The Ad Engineering team also repeated the experiment previously run on the legacy Ember.js application which was rendering placeholders for lazy loaded ads rendered inside the article content. Since the placeholder for this kind of ad was rendered once a request to the ad server was sent, we knew that there was still room for improvement. We came up with an idea to implement an experiment which we called ‘lazy requesting’. The idea was to inject incontent ads’ placeholders once the page loads and send consecutive ads requests when the user scrolls. Sending requests only when users scroll down has an indirect meaning for the viewability ads metric and thus for revenue. What is more, Fandom pays for every request sent to the ad server so sending requests for ads that probably will not be seen by users is a waste of money. Again, we ran the experiment on a selected group of wikis to see how these changes perform not only in terms of CLS but also for other important ads and user metrics. Initially, in this experiment the incontent ad placed the highest on the page was fast-loaded. However, because of this change, the ad was less viewable, so we had to change the implementation in a way that this ad was again lazy-loaded. The plot below presents the impact of the lazy requesting experiment on the CLS metric for given percentiles of page views with incontent ad impressions.
As you can see, this time we successfully decreased the CLS on pageviews with incontent ads.
Eventually, we moved the code which injected a placeholder for the top boxad ad on the new MediaWiki skin to the backend side. In the gifs below you can see the difference between rendering this placeholder on the frontend (left gif) vs. on the backend (right gif). This change was successful in terms of decreasing CLS but at the same time, it resulted in a viewability drop, and because of that, we didn’t roll it out to our global traffic.
The last placeholder we implemented was rendered on the backend for the bottom leaderboard ad which is an ad placed at the bottom of the page. This experiment was also successful but one should keep in mind that in the case of ads’ placeholders, the impact on CLS for a given page URL depends on the number of pageviews with a given kind of ad. It means that it will be bigger in the case of ads that are displayed on the majority of the pageviews like top leaderboard and less in the case of incontent ads which are injected into article content only if the article is long enough. That is all about ads’ placeholders but these are not the only experiments we conducted.
Ad products improvements
Except for so-called standard or base ads, there are ad products that are a combination of several different ad units. One of these ad products is Fan Takeover, which combines a couple of base ads like the top leaderboard and the top boxad and one special ad — highly viewable leaderboard (HiVi leaderboard). Not without a reason, this ad is called ‘highly viewable’ — it is displayed above the whole page content, pushes it down, sticks to a navigation bar, and stays in the viewport for a few seconds while scrolling.
To save visual stability of pageviews with this ad product the Ad Engineering Team had to rebuild it. The target was to decrease CLS and at the same time, keep other metrics on the same level. Fan Takeover was changed in a way the top ad renders inside the top leaderboard placeholder and sticks to the navigation bar when the user scrolls down.
This time we run an AB test to check how the new Fan Takeover implementation performs in comparison to the old one. However, it turned out that rendering the top ad in a place of the top leaderboard ad instead of displaying it above the navigation bar causes viewability and click-through rate (CTR) drop, especially on pages with infoboxes where the placeholder was placed below these boxes. Both of these metrics are important from the business perspective, so we needed to think about how to keep them on the same level while decreasing the CLS at the same time.
CTR decrease could be caused either by the fact that the ad is displayed below the viewport or by reducing layout shifts which resulted in fewer misclicks (or both). This may seem bad, but in fact, it’s not since it is a good user experience for readers and ads publishers. The readers don’t want the content to jump whereas the publishers don’t want fake clicks in their ads.
The viewability issue could be fixed by changing the implementation to stick the top ad of Fan Takeover to the navigation bar if the top leaderboard slot is not in the viewport. Another idea to improve the metrics was to always display the top ad of this ad product on the top of the web page even if an infobox is present. This way we can also unify the ad layout across different article types. At the time of writing this article, these experiments are still in progress therefore no results are available to be shown.
Other experiments
These were the most impactful experiments that the Ad Engineering Team was working on in the first half of 2021 but we had one more idea to improve web performance. The idea was related to our custom open-source library created to integrate all ads-related services and configurations called ad-engine. We have separate ad-engine bundles which load on different Fandom platforms and handle displaying ads, tracking, and ads’ targeting. As the Ad Engineering Team, we really care about displaying ads but there are cases when ads should be disabled. However, even in these cases, we used to load a full ad-engine bundle even if most of its code would not be run. Loading unused JavaScript code has a negative impact on web performance metrics like Total Blocking Time (TBT) or Time To Interact (TTI) so we decided to create a special ad-engine bundle dedicated to page views with disabled ads. This bundle handles tracking, integration with services, or requesting users’ consent in terms of data privacy needed even without ads. Before going live with this bundle, we ran performance tests on a special testing environment and we discovered that by decreasing the size of the ad-engine bundle by half, we can improve TBT by more than 40 ms, TTI by ~200–650 ms, and LCP by ~300–550 ms (these results apply to page views with disabled ads).
And last but not least interesting thing we discovered during our experiments with CLS was connected with CSS properties. I noticed CLS increase when ad loads and loading animation ends. Initially, I thought that it’s because of the animation but I dug deeper and I discovered that the reason behind CLS increase was setting max-width CSS property of the ad’s loader which was unset when ad loaded. According to this article, the reason behind it is that setting max-width forces a browser to
do a layout, which involves calculating the geometry (position and size) of all the elements affected by the change. If you change one element, the geometry of other elements may need to be recalculated. For example, if you change the width of the <html> element any of its children may be affected. Due to the way elements overflow and affect one another, changes further down the tree can sometimes result in layout calculations all the way back up to the top.
In the case of ad placeholders, setting max-width was advised by the Fandom Design Team and since the impact on CLS was not significant (increase by 0.004) we decided not to change it. However, I find this discovery quite interesting and I wanted to share this learning. According to this article, it is advised to use the ‘transform’ CSS property as it allows adding animations that don’t trigger layout shifts.
Fandom is a place where fans will find articles, experiences, and tools designed to help them share knowledge or go deeper into the worlds of their favorites and to discover new ones. For a platform like Fandom web performance, SEO, and user experience are extremely important. On the other hand, it can be challenging to indicate factors that have a negative impact on these dimensions. Sometimes what we expect is not what we get and this is why a data-driven approach is the best one in our case. In this article, I described the experiments and findings from the Ad Engineering Team perspective but these are not the only ones Fandom Engineering Teams performed. Improving Core Web Vitals on Fandom’s platforms is a cross-team effort and our journey does not end here. We continue to introduce improvements and monitor the data and we hope to create the best possible user experience for our fans.
