Why D7 ROAS might be the worst KPI to optimize towards since CPI

Nebojsa Radovic
Dec 3, 2019 · 7 min read
Photo by Jp Valery on Unsplash

Two months ago, during the UA Masters Summit in San Francisco, I interviewed some of the top UA (User Acquisition) professionals from the industry on the KPIs they use to optimize towards. Not surprisingly, D7 ROAS was the most common answer. In practical terms, this means that campaigns with the highest D7 ROAS get a larger share of the overall budget. At the same time, the lower-performing ones are getting scaled-down. As a result of this, the overall ROAS should improve. This has been the case pretty much since I started doing UA, and there are many reasons for that:

  • Media buyers don’t necessarily understand how predictive modeling works, and it’s significantly easier to report on and optimize towards one metric.
  • D7 ROAS paired with a multiplier for each product makes it easy to calculate what the ROAS for a specific campaign is. i.e., D7 ROAS for the campaign X is 8%, and the D7 multiplier for a casual game #1 is 15. The ROAS, in this case, for the campaign X is 120%.
  • Multipliers work well with smaller cohorts that don’t have a lot of install/engagement events that are mandatory for building a successful predictive LTV modeling algorithm.
  • 7-day cohorts are mature enough, so we could say with decent confidence that a particular cohort is good or not. D1 and D3 metrics are, in most cases, too early to tell.
  • It is, at the same time, early enough that there is a statistically significant number of users in the game that we can monitor in real-time and make decisions based on their behavior. D7 Retention Rate in games ranges from 10–20% in most cases.
  • Lastly, after seven days, it’s tough to impact user behavior as a UA team. Once the user completes the tutorial and starts engaging with live content, the ball is in the product and product marketing team’s hands.

However, times have changed, and optimizing towards D7 ROAS nowadays is equally risky as it used to be optimizing towards CPI targets a few years ago. Everybody used to do it, and then, due to install fraud and traffic mixing, media buyers had to up their game and move towards a more comprehensive KPI — D7 ROAS. It’s time to do that again. Here is why.

The problem with D7 ROAS multiplier

How D7 ROAS multipliers work

Generally, due to a pretty significant drop off in retention by D7 (80–90% of users leave the game in the first seven days), the LTV curve should start flattening after that point. Therefore, it is easy to model the remaining part of the curve can like a linear function. This means that using a multiplier to estimate the remaining portion of the curve shouldn’t be too inaccurate, especially if you are driving thousands of installs daily. In reality, however, this approach only works if you are buying from one channel in one country and platform using one optimization model. Things get significantly more complicated when you take into account different traffic types and optimization models, especially ROAS optimization ones.

How different traffic types and optimization models impact the LTV curve?

Acquiring users (for mobile games at least) at scale teaches you a few things:

  • Facebook and Google offer sophisticated targeting capabilities that help you find the high cost / high intent users.
  • Incent channel users are quite the opposite — low cost / low intent users.
  • Rewarded video channel users are somewhere in between. They are coming from other games, which makes them a somewhat qualified audience. Thus, we can call them mid-cost / mid-intent users.
  • Users from different countries behave differently. i.e., payers from lower GDP countries often behave similarly to payers from Tier 1 countries that come from incent channels.
  • There is a significant difference in performance between Android and iOS users in terms of scale and quality.
  • There is a significant difference in performance between different device models. i.e., lower-end and higher-end Android devices.
  • The performance varies significantly at different levels of spend.

When you take all these into account, there is simply no way that the “one size fits all” approach to modeling LTV could be used. Each combination of these (channel/platform/optimization model/country/device) has a different shape of the LTV curve, and as such, cannot be estimated using the same multiplier. If we look at the channel groups only (FB/Google, Incent, Rewarded Video), the LTV curves for each of these channel groups, from my experience, look much closer to the ones in this image.

If the ideal scenario covers rewarded video and Facebook/Google mobile app install users where the multiplier approach can be used successfully, using the same multipliers for ROAS based bidding and incent traffic usually doesn’t do the best job.

What is so specific about ROAS optimization models?

ROAS optimization models, Facebook’s Value Optimization (aka VO), and Google UAC Target ROAS (tROAS) are, in many ways, a dream come true to mobile marketers. When they were first introduced, all the major UA metrics went up significantly. FB VO campaigns have, quite often, 5+ times higher payer conversion rates than rewarded video or MAI (mobile app install) campaigns. Retention and ARPI are following the same trend. So does the cost.

With VO, Facebook built a sophisticated solution for game developers that enables them to go after the most coveted set of users — payers in other games, including high paying users, aka whales. These are also known high-intent users that play and spend in multiple games and progress through in-game content significantly faster than other players. Given the high early payer conversion rate, the number of transactions they generate in the first few days is high enough so that Facebook can move away from install and event optimization down to ROAS optimization.

This is great, right? Not necessarily. It is great if all other metrics remain unchanged. In particular, payer retention and cost per payer. In that case, the remaining part of the curve looks the same, and higher D7 ROAS means, indeed, a shorter payback.

Hyper-targeting = Hyper-competition

One more thing that is important to mention is that this works only if market conditions remain the same meaning — the same number of advertisers goes after the same number of users.

It is likely that this is not true, and it is probably the most significant change in the market. With more sophisticated optimization models, advertisers now finally have a way to hyper-target only the users they want. The same number of advertisers is going after a significantly smaller group of users which impacts two things:

  1. The cost is going up, and advertisers are paying a premium to acquire higher-quality users. This increases the risk profile of doing UA in the first place.
  2. Payers are getting bombarded by ads (which explains why IPM is lower for VO campaigns) from other games who desperately want them contribute to their economies. Everybody wants a piece of the whale pie.

As I mentioned in the previous paragraph, these payers play and spend in multiple games and progress through in-game content significantly faster than other players. Combined with the fact they are also more exposed to other content, this pretty much leads to the conclusion that the payer retention has to suffer. Therefore, since the cost is up and long-term payer retention lower, the ROAS curve flattens sooner than predicted, and money pays significantly later than initially predicted with multipliers.

For illustration, incent is quite different. It’s all about whale trawling and finding low cost/low-quality payers hoping that one of them will be a bigger payer and pay for the whole cohort. Thus, early metrics for the incent users are generally pretty low (due to low intent). However, due to the low cost, any later transactions have a bigger impact on the LTV curve than what would be the case with high cost/premium sources.

The risk profile of acquiring incent traffic is fairly low if you understand how to do it, define its benchmarks, and focus on lowering the cost/increasing long term engagement by moving the payout event further.

Does this mean that the industry should completely move away from optimizing for D7 ROAS?

No. There are quite a few benefits of using D7 ROAS as I outlined in the first paragraph. The challenge here is not really with Facebook or Google either, but with using a single metric to optimize towards. It’s just that, in this case, that’s D7 ROAS and FB VO, and Google tROAS helps us get there easier. The comparison to CPI from the headline is quite adequate as with more sophisticated algorithms, we just moved the payout from CPI to D7 ROAS, but the approach is quite the same — it’s optimizing to one single metric.

If a company is looking to do UA at scale, it should never make decisions just based on one metric but try to understand the products they promote deeply and how different marketing levers (platform/country/channel…) impact the performance. The learnings should be implemented in LTV modeling, and the UA team should work closely with product and data science teams to understand how each one of those impacts the performance. Or, in case the company doesn’t have a large data science team, it should at least focus on defining custom models for the largest channels and move away from a “one-size fits all” solution.

The output of both solutions (using predictive LTV models and custom multipliers) does lead to one — more comprehensive — goal, and it’s shortening the payback window, which will allow you to recoup the money sooner and spend more of it in the future. All other metrics should be just proxies for that.

Nebojsa Radovic

Written by

Everything mobile growth — user acquisition, retention, monetization. Ex-Nordeus, Ex-MZ. Now Performance Marketing @ N3TWORK, Inc. aka > @eniac

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade