Modeling the Facebook Ad Fatigue Phenomenon — Part 1

Nizam Makdoud & Marc Caillet

Introduction

Most video content creators on social platforms need to monetize their work to make a living out of it. Unfortunately, it is hard for many of them to thrive in the Creator Economy, the cornerstone issue being the need for more visibility of the creator’s content. The game is highly competitive, and getting enough people to interact with their content is quite challenging. Growing a loyal community by relying solely on the social platform’s recommendation algorithm usually takes too long. But what if it were possible to speed up this growing process?

One of Jellysmack’s ambitions is to help video content creators quickly grow their community and thus quickly generate revenue out of their content. How? By leading precisely targeted advertising campaigns while optimizing their cost.

But beware! A potential pitfall lurks around the corner and could very well make you waste your money!

Barbara

Take Barbara, for example. Barbara is a Youtube star. She wants her content to reach new people. So, Barbara signs up on Facebook and starts publishing her video content there, hoping that she would reach and engage with more people and generate new revenue. At the time of writing, Barbara’s Facebook page must have at least 10,000 followers and cumulate at least 600,000 minutes of views in the span of 60 days to be monetizable. But, despite keeping publishing quality video content on her new Facebook page, Barbara struggles to grow her community. The number of views stays desperately low, the number of engagements is way lower, and Barbara’s Facebook page still does not generate any revenue. Her attempt to thrive on another platform is not as smooth and seamless as she had wished. As frustrating as this experience can be, Barbara keeps on posting new video content. Slowly, she gains new followers. But the process takes too long a time. It is not rewarding. At some point, Barbara might start to think that it may not be worth the effort.

Barbara’s experience is all too common. A community hardly spreads from one social media to another. Take any video content creator thriving on Youtube. They would have to start building a community from scratch should they try and expand their reach on Facebook. But the broadness of the non-followers audience is a function of the number of followers. Moreover, the fewer followers, the less information available to Facebook to precisely target users of the social platform who might be interested in Barbara’s video content. Hence a significant amount of time is required to grow a community.

What Jellysmack can do to help Barbara

Since we believe in the quality of Barbara’s video content, attracting a new audience is a question of reaching new people who might be interested in her content.

Most social media allow for advertising campaigns. They are primarily intended for brands to promote their products and make the social media’s users buy them.

To help Barbara quickly grow her community on Facebook, Jellysmack would create advertising campaigns. The promoted product here would be Barbara’s video content. The goal would be to expose carefully targeted Facebook users who may show interest in Barbara’s content to her videos. The core idea here is that, by paying for views, we could somewhat control the view rate and likely speed up the growth of Barbara’s community.

The curse of ad fatigue

But successfully managing advertising campaigns is challenging. Coming up with the best mix of targeted audience, promoted video, and budget is a complex task:

• The targeted audience’s interests need to match the content of the promoted video. A user likely to be interested in the promoted video they are exposed to is also likely to become a new follower of the promoted video’s creator.
• We do not want to bore the targeted audience with too high exposure to the promoted video. A bored audience poorly translates into new followers. Interestingly, even if the audience shows interest initially, it will inevitably get bored at some point should the exposure to the promoted video last too long. This phenomenon is called ad fatigue [1]. Long story short, an advertising campaign that starts out to be promising in terms of newly acquired followers is at risk of poor results at some point if we do not carefully manage it.

So, being able to anticipate ad fatigue is key to the success of an advertising campaign. It would be a critical asset for our advertising campaign managers:

• We could warn them that ad fatigue is about to show, and they should stop spending on the current targeted audience.
• Or, they could be confident that they are in the clear and keep spending on the current targeted audience.

This article is part 1 of a series of two dedicated to how we modeled Facebook ad fatigue. On our way to modeling this phenomenon, we will discuss the limitations of traditional data-driven approaches in machine learning.

Background

The inner workings of Facebook advertising campaigns

Like most other social media, Facebook gathers a lot of information about the people who interact on the platform, content creators, and consumers alike. It uses this information to profile its users and their interests. In advertising campaigns, this knowledge is essential because it enables advertisers to precisely target a particular subset of the Facebook audience. From an advertiser’s viewpoint, it allows for maximizing conversions at a much lower cost. It gives Facebook an unfair advantage compared to traditional media, such as TV or radio, which cannot allow for such a targeting precision level.

So, what do you buy — or rather bid on — when launching an advertising campaign on Facebook? Impressions — displays, in other words — of the content you promote on your target audience’s feed.

But then, multiple advertisers could target the same audience simultaneously. Each time there is an opportunity to show a promoted content to a user who belongs to the targeted audience, Facebook auctions the impression, and the better bidder wins the impression [2].

Showing irrelevant ads to its users would lead to an ad fatigue phenomenon at a global scale: fewer and fewer users across the whole social platform would care to watch advertisements that would pop up on their feed. To avoid this situation, Facebook is cautious about which ads it shows to whom.

Figure 1: Factors that weigh in auctions for impressions.

As depicted in Figure 1, multiple factors come into play when Facebook decides on the winner of an auction [3]:

• Ad Quality
  Facebook values quality video content because users are more likely to show interest in high-quality ads. A poor-quality ad is highly likely to lead straight to ad fatigue. So, the better the quality of the ad, the more chance of winning the auction.
• Fit between ad content and targeted audience interests
  The ad’s video content should meet the audience’s interests to engage them in the ad. A poor fit would lead you straight to ad fatigue. So, the better the fit between ad content and targeted audience interests, the more chance of winning the auction.
• Budget
  Setting up the right amount of money is far from being trivial. You risk being outmatched at each auction if the amount is too low. Should it be too high for too long, you will soon experience ad fatigue even if the quality of your ad is the best and the fit between ad content and targeted audience interests is perfect. Because, at some point in time, you are likely to have reached most of the Facebook users your ad could likely have converted to new followers. And the few remaining ones get harder and harder to reach.

These factors show that high-quality video ads targeted at audiences whose interests fit the ad content together with a smart investment strategy are the way to postpone the emergence of the ad fatigue phenomenon. They are necessary conditions for a fruitful and budget-optimized advertising campaign.

How Jellysmack grows a community using Facebook advertising campaigns

As mentioned in the introduction, Jellysmack uses Facebook marketing campaigns [4] to promote the video content of creators who entrusted us with growing their community.

How do marketing campaigns help grow a community? First, some users who belong to the audiences we targeted are shown our promoted video content. Then, each one can choose to view the video. Finally, viewers automatically become followers of the video’s creator if they hit the like button (c.f. Figure 2).

Figure 2: From advertising to growing a community.

So, we grow a community by acquiring the biggest number of followers through advertising campaigns at the lowest cost possible. Our success metric is the Cost Per Like (CPL). The CPL can also be seen as the cost per newly acquired follower since every non-follower who hits the like button on promoted content is automatically converted into a follower.

Now, let’s go back to Barbara for a short while and say she decided to entrust Jellysmack with growing her community on Facebook. Since Barbara makes videos about her travels worldwide, a dedicated campaign manager must target audiences interested in traveling. Because of ad fatigue, the campaign manager could not focus on just one audience. They would need to invest money in multiple audiences. For example, the campaign manager would target not only people interested in traveling but also people interested in countries Barbara has visited or people interested in landscapes or food from around the world. Since Jellysmack has published about 800,000 videos on many different topics, the campaign manager could also pick lookalike audiences to reach Facebook users similar to viewers of our other travel video creators’ Facebook pages. Of course, the promoted video content would need to be fitted to the specificities of each audience. Finally, the campaign manager would have to make the best out of their budget by spending on promising audiences while stopping spending on audiences for which they anticipate ad fatigue.

It is important to note that part of the budget is considered an advanced payment on future revenue Barbara’s Facebook page would generate once monetizable. Consequently, we are accountable to Barbara for every dollar spent on growing her community.

The example of Barbara illustrates the steps an advertising campaign manager needs to take to create a successful campaign:

1. Targeting the audiences with the best potential. The people who manage the campaign need to identify the audiences whose interests best match the creator’s content. Understanding the reason for the quality of the creator’s content helps to select an adequate audience.
2. Choosing the right ad for each selected audience. Because each audience might be interested in a creator’s video content for different reasons, advertising campaign managers need to personalize the promoted content.
3. Managing the campaign. Once a budget has been allocated to the advertising campaign, the campaign manager who is in charge of this campaign must split it among the different audiences they choose to invest in. As the campaign progresses, the manager needs to update the money distribution. They would invest more in successful audiences. They would stop investing in audiences for which they anticipate ad fatigue.

Figure 3 highlights those three steps: targeting the audience with the best potential (Audience), fitting the ad to the target audience (Ad), and setting a budget (Budget). All three boil down to setting variables over which we have control. Hence, we can optimize the audience’s targeting and the ad’s fitting to maximize the number of newly acquired followers for a given budget.

However, we do not have any control over the competition for the same target audience (Advertisers). This variable directly impacts the CPL, though, because the more competitors, the higher the price for an impression.

Figure 3: Campaign managers control the ad’s fitting, audience targeting, and budget setting.

Once the campaign managers have set the ad, audience, and budget variables, Facebook daily bids on our behalf for impressions of our promoted video content to a part of the target audience. We then observe our ad’s impressions, views, and likes (c.f. Figure 4). It allows us to compute impressions-to-views and views-to-followers ratios, which we aim to keep as high as possible.

Figure 4: From an impression to a newly acquired follower.

The more budget invested in a thriving audience, the more new followers. This is true up to a certain point. Every campaign manager knows that ad fatigue awaits. None of them knows when it will strike.

Helping advertising campaign managers better optimize their campaigns

To make the best out of a budget allocated to an advertising campaign, i.e., to maximize the number of new followers who will be acquired thanks to the campaign, anticipating the emergence of the ad fatigue phenomenon is key.

As mentioned above, ad fatigue emerges when an audience gets bored with an ad they are shown. This phenomenon is visible when a few audience members like or even only view the ad. It translates into a diminishing number of newly acquired followers while the cumulated amount of money spent on the audience keeps increasing.

An audience gets bored very quickly if they are shown an ad with nothing to do with their interests or an ad of inferior quality. But we assume they will unlikely see such an ad pop up on their Facebook feed because the advertiser might never win the auction.

The intuition behind the emerging ad fatigue phenomenon we usually observe is simple. An audience is composed of a finite set of people. As the advertising campaign progresses and more and more people become new followers, the number of people who are yet to be converted naturally diminishes. As their number decreases, those non-followers get harder to reach. So trying and reaching them would require spending more money.

Consequently, the CPL, our metric for success, the one we aim at minimizing, would increase. This is why we need to stop investing money in an audience as soon as we anticipate the emergence of ad fatigue.

Figure 5: A real-life example of newly acquired followers over advertising campaign investment in different audiences.

Figure 5 shows real-life newly acquired followers over advertising campaign investment in different audiences. It depicts how a campaign manager tries and infers which audience has the best conversion rate (the steepest slope in the image, assuming linear modeling between spending and conversion). We observe that investment in all audiences but the pink one has been stopped at some point. For example, investment in the blue audience stopped once it reached a $150 daily spending.

Let’s now focus on the brown, purple and blue audiences (c.f. Figure 6). Although quite promising, we observe that the purple one could have been topped by the brown one, in which the investment was prematurely stopped. The blue one ran out of steam early on. Investment in this audience should have been stopped earlier.

In many cases, the campaign managers perform well enough. But in other cases, they anticipate ad fatigue prematurely and stop investing in thriving audiences, or they do not anticipate it at all and keep investing in poorly or mildly performing audiences. In practice, they stop investing in an audience as soon as they feel that ad fatigue might emerge, but they never observe it. There is room for improvement and optimization. The campaign managers could use a decision support model to help them better anticipate this phenomenon and make more profitable advertising campaigns.

Figure 6: Investing in the purple audience was a rather good call, but investing in the brown one might have been better. Investment in the blue audience should have been stopped way earlier.

Modeling the Ad Fatigue Phenomenon

Modeling the exact advertising process on Facebook is complicated, if not impossible. As discussed in the previous section, we control some variables (ad fitting, audience targeting, budget allocation), but others, such as the competition, are out of our reach. Not only do we have no control over them, but we also cannot observe them. Since the ad fatigue phenomenon we want to model is strongly related to advertising on Facebook, it is doubtful that we will be able to model it accurately.

Consequently, every model we will come up with will be wrong, meaning it will not be the most accurate depiction of this phenomenon. Hopefully, one of them will be helpful and allow the campaign managers to optimize their investments better.

Before we rush into devising a model of our own, let’s try and test some data-driven models using traditional machine learning approaches, from simple to more complex ones.

Linear Models

In the previous section, we used a linear regression model to show that the campaign managers’ investment strategy is not optimal (c.f. Figure 5). Could we use it to model the ad fatigue phenomenon and anticipate it?

Each line in the graph shows the performance of an audience. The slope represents the conversion rate. The steeper the slope, the higher the conversion rate and the happier the campaign manager is about their campaign.

This model fits our intuition that performance varies according to the target audience pretty well.

Now, what would this model recommend? It would advocate spending on the audience with the steepest slope, thus with the highest conversion rate, which seems a pretty rational move. That is, if we were evolving in a world with infinite growth. Indeed, with this model at hand, extrapolating future performances would show no decrease in newly acquired followers as the spending keeps increasing. This analysis contradicts our expert knowledge about the ad fatigue phenomenon, which shows decreasing performances as the investment keeps going (c.f. Figure 7).

Figure 7: Estimated conversion rate according to expert knowledge vs. overly optimistic linear conversion rate.

A linear regression model is simply blind to ad fatigue. Although it correctly models past performances, it cannot predict the emergence of this phenomenon. Yet, it is helpful in that it highlights the audiences that perform the best at the moment.

First model. A very wrong one. Unable to predict ad fatigue, but practical in some way.

More Complex Models

So, linear models do not suit ad fatigue modeling. How would more complex models do? How would an XGBoost-based model do, for example?

XGBoost is a sophisticated algorithm that implements a scalable Gradient Boosting Decision Tree algorithm. Gradient Boosting Decision Tree trains iteratively many decision trees to predict a target label. It first trains a set of weak learners (the decision trees). Then, an additive model aggregates the output of each decision tree in this ensemble.

XGBoost is known to perform exceptionally well in contests such as Kaggle, especially for problems involving tabular data.

Now, let’s try and use XGBoost to model ad fatigue. First, we fit XGBoost with inputs such as budget, audience name, and metadata about the overall campaign and the creators. Second, we observe what would be the model’s prediction outside the range of data we fitted it with.

Figure 8: XGBoost trained over past data and extrapolating.

Figure 8 shows that XGBoost does a great job at making regressions over past data. It also shows its inability to extrapolate! Indeed, it predicts that a budget larger than $200 would result in no additional followers, the number of which would remain constant at about 1,400. This observation confirms what others have previously analyzed.

In retrospect, this is no surprise since XGBoost predictions are based on summing values computed at the tree leaves level. It means that XGBoost cannot predict anything outside its training range.

As mentioned in the Background section, the campaign managers stop investing in audiences before the ad fatigue phenomenon emerges. They often stop investing way before the phenomenon appears because they do not want to risk wasting money. Consequently, we have no data about ad fatigue emergence over which a model could be trained. Given our dataset, XGBoost was doomed to fail at modeling this phenomenon.

The same applies to Neural Networks of any depth. There is little to no chance that such a model would be able to learn a phenomenon that is nowhere to be observed in the data.

The issue we face is simple. Multiple functions can fit our data. In terms of extrapolation, they all represent different possible futures. As such, they are likely to give conflicting predictions, as shown in figure 9.

Figure 9: Illustration of extrapolation uncertainty when models are fitted to a limited range of data. The left panel in the figure represents what the response curve might be if the advertiser had spent in those ranges. The middle panel would be what is available to the modeler due to the advertiser spending only in a limited range. The right panel shows four fitted response curves. (Source: Challenges And Opportunities In Media Mix Modeling [5][6].)

Figure 9 clearly depicts the situation we face: our dataset only contains data for a specific range of spending. Within this range, no ad fatigue emergence can be observed. Within this range, multiple functions would be a good fit. Within this range, linear, quadratic, and logarithmic models would perform similarly. Outside of this range, they all would have highly different behaviors.

Given our dataset, it is now clear that data-driven only approaches as a whole cannot model ad fatigue.

Beyond Data-driven Approaches

So far, we have shown that neither a linear model nor more complex models can adequately model the ad fatigue phenomenon.

The linear model failed because the underlying hypothesis makes no sense (an infinite budget would allow for an infinite gain of new followers).

More complex models failed because our dataset is limited. It is composed of data that lie within a specific range of spending and has no data showing the emergence of the ad fatigue phenomenon. Consequently, many different models could fit those limited data, making any extrapolation unreliable.

One might suggest that we build our own ad fatigue dataset over which we could fit a model. Building such a dataset would require spending substantial money on advertising campaigns. It would cost Jellysmack a lot of money. It would also be costly for Barbara and our other creators because part of the budget we invest in advertising campaigns is an advanced payment on their future revenue. Requiring them to agree to significant financial efforts to build our dataset would be unrealistic. It would also be unethical and go against our primary goal of growing our creator’s community as fast as possible at the lowest possible cost.

Traditional machinal learning approaches work well when data is plenty, and the data generative process is complex and unknown. Our data is scarce. So, in our case, traditional machine learning approaches are of no help.

Let’s take a step back and ask ourselves where we got how ad fatigue emerges from, if not from our dataset. The answer is straightforward: we got this from domain expert knowledge. What we need to do is to embed this knowledge into our model.

Bayesian modeling makes for the perfect candidate: it suits scarce data; it leverages Bayesian inference to devise a model and embed our knowledge about ad fatigue emergence as priors.

Stay tuned for part 2 in this series of two! This second part will detail how we modeled the Facebook ad fatigue phenomenon in a simple, interpretable, yet powerful way.

References

[1] About creative fatigue guidance in Meta Ads Manager, Meta Business Help Center.

[2] How Does Facebook Ads Bidding Work and How Much Do Ads Cost, Buffer, Medium Article, 2017.

[3] About Ad Auctions, Meta Business Help Center.

[4] Getting Started with Facebook Ads, Buffer, Medium Article, 2017.

[5] Challenges And Opportunities In Media Mix Modeling, David Chan & Michael Perry, Google Inc., 2017.

[6] Bayesian Methods for Media Mix Modeling with Carryover and Shape Effects, Yuxue Jin, Yueqing Wang, Yunting Sun, David Chan and Jim Koehler, Google Inc., 2017.