The Biggest Ad Monetization Lie You Overlooked

Not a lot of things are worse than someone stealing money right out of your bare hands, right?

Unfortunately, that is what a lot of ad networks are doing.

CallApp, a caller ID app with 100% free features, is generating revenue from ads. With this being said, measuring our ad monetization partners is of the utmost importance. Along with measuring them, we have to mediate them and set up the right waterfall. CallApp usually uses about 6–7 ad networks in total, with every now and again giving a new network (that support native ads) a try. This was exactly the scenario with a new ad network we were trying (called ‘Network X’ from now on). The integration took a little bit more than expected, but since this network was native-ad focused, we believed it would be worth it.

Launch day finally came and we started to track the network’s performance. Within a few days, I noticed something was off. Network X was showing decent (yet not incredible) eCPMs. But overall, revenue was dropping. How could this be? This called for an investigation.

I deeply examined a single GEO, for one week, in which Network X got some traction. It was important to isolate a stable GEO where no other changes took place that week.

Step 1: Ad requests

Ad requests usually represent the ad network’s position in the waterfall. They are also a clear sign of who get’s the ‘best’ traffic (and the first impression). This is the case if your waterfall is set up right, with the right price floors and priorities.

Taking a closer look at my waterfall, I found these numbers:

Ad requests show no sign of an issue — so far so good. Let’s move forward.

Step 2: Filled requests & Fill Rate

The fill rate is the rate at which ad networks accept and fill ad requests, rather than refusing and passing it along to the next one in line. Sometimes even a 5% fill rate is fine, but usually it suggests a problem — either the network doesn’t have sufficient demand in that GEO, there is too high of a floor price, a technical error or something else.

Nothing fishy so far. Let’s dig a bit further.

Step 3: Impressions

Impressions are the basic metric the ad networks and publishers use to monitor their performance. Solely using this metric as a performance indicator may be misleading, as we will soon learn. To get to the bottom line of this issue, measuring impressions will play a vital role.

This doesn’t look right.

Out of 8M filled requests, only 1.1M were actually counted as impressions. The percentage of shown impressions from the overall filled ad requests is known as the ‘show rate’. It is important because the eCPM is calculated from visible impressions.

To be sure, let’s add the ‘show rate’ and compare performance.

Step 4: Show Rate

This makes perfect sense now.

An ad request unfilled goes to the next in line. When using any type of mediation, there’s no harm in letting a request go. Ad networks know that, so most of them do not accept all ad requests (even if they have an ad to fill). However, when an ad request IS filled, it’s game over and all in their hands now.

On the other side of things — when it comes to revenue, the app publisher doesn’t care if the ad was counted as ‘shown’ or not ‘shown’. We want to maximize revenue, that’s all. Impressions, whether visible or not, are not relevant for publishers when comparing performance. Filled ad requests, therefore, represent the most significant point in waterfall management — the point from which the ad networks take responsibility for the ad request. It only makes sense to measure performance using that parameter.

Step 5: True eCPM

Let’s check a more accurate kind of eCPM — one that considers filled requests (and NOT impressions):

Let’s keep it real — in many cases, a high show rate (over 70%) is unreachable. Some ads are loaded but not shown, as you need to scroll down the page to see them. Some take time to load. The problem is — each network has a different way to count an impression. Usually it’s around 2 seconds when an ad is visible to the user, but it does vary. And by vary, I mean possibly abused.

Step 5: Making Sure

But wait, what if this is simply a technical error? We considered it, so we decided to create a very simple test. A new ad-placement, with the same ad units and GEO, gave it 1st priority and ran it 153 times. Each time the ad was shown, we made sure it was 100% visible for over 5 seconds. This is more than enough for any ad network.

Results:

• 153 ad requests
• 153 filled requests (100% fill rate)
• 30 impressions (~20% show rate)

At this point, it’s fair to say that Network X, who is a very well-known ad network within the mobile industry, is cheating us. Shame. Good reputation, good customer service, always paid on time… Nevertheless, this kind of thing speaks for credibility. We will not work with networks that lie so blatantly, even if all their employees are professional, supportive and helpful (which they were). Following these results, and once we confirmed no technical issues were involved, we decided to remove the Network X SDK from our app, completely and indefinitely.

Conclusions:

I learned a great deal from this.

First, trusting the people you work with is fine, but always trust the data. Data rarely lies.

Second — in the ad waterfall, or at any point where ad networks can be compared, impressions and show rate can easily be manipulated to show a better eCPM and performance. Therefore, when working with ad networks you need to demand the entire data from the ad funnel — that includes requests, filled requests and Show Rate. Comparing eCPM from filled request is the best way to find out who truly has better performance. If all networks have a similar show rate — then you can move to compare standard eCPMs.

In the mobile ad industry, eCPM is considered the holy grail when tracking performance. That’s where everyone looks. As it turns out, there’s a deeper layer to investigate. Don’t be fooled by appearances. Remember you can’t judge a book by its cover.

(Photo by Christine Roy)