What is probabilistic attribution? (and every other iOS14 question you were afraid to ask)
With the goal of educating ourselves and industry stakeholders on the implications of iOS14 and the privacy announcements, we started a living FAQ. This will be updated as more questions are asked, and more answers are revealed. If you’d like a question answered please email us at firstname.lastname@example.org
1. What does iOS14 mean for measurement of user acquisition campaigns?
At a high level, Apple made several privacy-centric announcements with their iOS14 with respect to the measurement of user acquisition:
- To access the device “Identifier For Advertisers” (IDFA), an app developer would need permission from the end-user through an App Tracking Transparency (ATT) popup in every mobile application used by that device
- Where users didn’t give permission to access their IDFA through the ATT popup, Apple has made available SKAdnetwork, an attribution solution for iOS14 that has the following ramifications for measurement of performance marketing on iOS:
- User-level campaign and channel attribution data will not be available
- Network, campaign, geo, creative, source app and placement will all be missing from the attribution data at the user-level. Only anonymous: campaign level installs, source-app and some down funnel events will be reported
- Campaign performance data for down-funnel events will be restricted
- Revenue and in-app events will be significantly limited. This move is intended to ensure no back-engineering of the SKAdnetwork solution which would allow advertisers to directly attribute a user to a campaign
This announcement completely disrupts mobile measurement. Mobile app marketers will no longer be able to attribute app installs to a channel or campaign and will thus not be able to track conversion rates and ROAS at the channel or campaign level. The current, common practice of adjusting campaign bids and budgets based on D7 ROAS or cohort ROAS curves at the channel or campaign level will no longer be easily accessible for advertisers.
2. What is the App Tracking Transparency (ATT) popup
This is a popup called by the app developer when they wish to gain access to the device IDFA.
Once the user allows tracking on this popup, Apple will enable access within this application to the device IDFA. This can then be used for user acquisition, retargeting and ad monetization purposes.
3. What is deterministic attribution?
Deterministic attribution is the exact matching of an app install to an advertising campaign at the user level. When an app install is deterministically attributed to a campaign, there’s an assumed 100% probability that the install campaign came from a specific campaign.
From the Wikipedia definition:
“In mathematics, computer science and physics, a deterministic system is a system in which no randomness is involved in the development of future states of the system. A deterministic model will thus always produce the same output from a given starting condition or initial state.”
Mobile Measurement Partners (MMPs) are able to assign 100% probability because the IDFA of a user clicking (or viewing) on an ad is exactly the same as the IDFA sent by the MMP SDK when the user opens the app after the install. MMPs use data engineering efforts to ensure that the receipt from the advertised app of the install event falls within the lookback window, and match back that install event to the advertising campaign that delivered the last click or view on the ad. The lookback window is a period of time after a click where the campaign that drove the install will get credit for its delivery.
4. What is probabilistic attribution for iOS14?
Probabilistic attribution is the process of assigning campaign membership probabilities to a user based attributes and behavior of that user. Unless a user shares their IDFA on iOS14, we cannot assign a 100% probability that a specific campaign drove an install, Therefore there is a set of probabilities that the install was driven by one or more campaigns.
In the context of iOS14, the following user-level data sets are available:
- Anonymous user-level in-app data: customer-generated user ID, revenue data, in-app events
- SKAdnetwork data: postbacks containing campaign ID, source-app-id, ConversionValue (more about this later)
- Deterministic attribution from MMPs: opt-in iOS14 users and users identified by other means within Apple’s terms of service
- Ad network reporting data: campaign ID, impressions, spend
Probabilistic attribution does not use any personally identifiable information about a user, as by definition that would allow for matching of an install to a campaign. For this reason, and in the case of iOS14, the data used shouldn’t use IP address (more about this later)
5. What is ConversionValue?
ConversionValue is provided by Apple within the SKAdnetwork framework to power probabilistic attribution. To protect user privacy, Apple restricted the amount of post-install data that could be reported against an SKAdnetwork campaign.
The following points are true about ConversionValue:
- ConversionValue is a single value and should be a proxy for LTV
- ConversionValue isn’t reported against a specific install ID and is therefore anonymous. ConversionValue is reported against a campaign ID.
- It should be sent by the app to Apple once the app developer has collected enough information about an app install to predict the LTV
- ConversionValue can only be sent once to Apple.
Examples of uses for ConversionValue:
- Revenue: reports the revenue generated by the user within the app
- Retention: reporting the retention rate of the user. For this the user needs to open the app over multiple days
- In-app events: reports a series of in-app events that the user completed
- Predicted LTV: reports the predicted LTV for the user based on revenue, retention and in-app event (engagement)
We can see from these examples the optimal use of ConversionValue is to use predicted LTV as it encompasses all other possible uses for ConversionValue.
6. Why do I have to use a probabilistic attribution model? Why can’t I just use ConversionValue to optimize campaigns?
Optimizing only to ConversionValue is another possible approach to optimizing campaigns. In this case, the advertiser wouldn’t model the predicted revenue that campaign drove, but only model the long-term ROAS of users/cohorts as a function of ConversionValue, either by encoding predicted LTV directly into ConversionValue or by building a model that maps ConversionValue to LTV at some time horizon. The model can be built using historical user data as long as the definition of ConversionValue is consistent for those users.
Ideally, the ConversionValue (whether based on predicted LTV or some other behavioural signal) acts as an early signal for ad networks to optimize against. Later evaluations of ROAS for particular campaigns or channels should incorporate more matured behavioural data, as opposed to operating on just an early predicted LTV bucket or conversion event. This is where probabilistic attribution becomes critical — so that advertisers can properly distribute their most up to date revenue projections amongst campaigns and channels.
The diagram below shows the various datasets available to advertisers in a post-iOS14 world:
- SKAdnetwork postbacks which deliver measurements of installs and ConversionValues associated with campaigns/source-apps but is fully anonymized and does not have any associated user IDs
- Traditional network-reported metrics (mainly spend, impressions and installs)
- Anonymized (user level) behavioural data, which will continue to be a very rich dataset encompassing all users’ behaviour throughout their lifetime of using that app
- Deterministic attribution from MMPs: this data is missing from the below diagram. This data is not used in the probabilistic attribution model, apart from to assign a 100% probability to the opt-in installs and reduce the number of installs necessary to assign a campaign membership probability.
Simply optimizing against ConversionValue postbacks neglects valuable data that comes in as cohorts mature and allow for updated LTV projections. A probabilistic attribution solution should be built to leverage all datasets as best as possible, to provide the most accurate estimates of asset-level ROAS.
7. Why is a top-down (MMM) approach to iOS14 attribution suboptimal?
Another alternative measurement approach on iOS14 is a top-down model or a “Media Mix Model”. A media mix model is an alternative advertising measurement approach to last-click attribution and uses probabilistic methods versus the deterministic model of the last click. In the paradigm of iOS14, the model would attempt to quantify the incremental return on ad spend (ROAS) from any given channel within the context of a portfolio of channels. A critical piece of this model would be the inclusion of the incremental revenue from organic installs as a result of paid user acquisition.
When attempting to quantify the incremental returns of any given channel, our first task would be to model the absolute returns of that channel. Although SKAdnetwork offers limited capabilities, it does report anonymous install counts and ConversionValue. Both pieces of data are sufficient and critical to model the predicted returns from any channel. Due to the availability of this data at the campaign level, it’s better to take the bottom-up approach than to throw out critical data that can help us model channel or campaign performance. It is, therefore, optimal to continue executing a bottoms-up approach to attribution than to move wholly to a Media Mix Model.
That said, there are use cases for an MMM-like approach to model incremental organic installs and revenue from paid acquisition (organic lift) at the channel level. But these approaches often run into data sparsity issues for smaller channels. It is unlikely that they can easily be extended to the campaign-level.
In short, MMMs ignore valuable signals that can build a bottom-up view of performance, such as SKAdnetwork conversion values that give some idea of ROI at the campaign or channel level. Additionally, MMM as a statistical approach needs a minimum install volume that’s typically unavailable at the campaign level.
8. Why is last-click attribution here to stay?
Last click attribution only rewards the last campaign for the app install but it’s likely that a user viewed or clicked on more ads than just the last ad they clicked before they installed the app. Self attributing networks (SAN’s) don’t report user-level click or view data to MMP’s unless they wish to claim credit for the install. This gives an incomplete view of the user’s journey to install and makes multi-touch attribution on mobile impossible. SKAdnetwork further compounds this problem. Not only is SKAdnetwork a last-click attribution model, but it also removes the ability to track impressions across ads and ad networks making it impossible to collect the necessary data to perform multitouch attribution. At present, the best we can hope for is a statistical estimate of last-click attribution.
9. What is fingerprinting and why is it flawed?
Fingerprinting is a solution provided by Mobile Measurement Partners where the MMP SDK is unable to access the device IDFA. This is either because the ad network can’t access the IDFA (ie — if the ad is on mobile web), or because the user has restricted access to the IDFA through Limit Ad Tracking (LAT).
MMP’s create an ID when a user clicks on a mobile ad using device data such as IP address, device name, device type, OS version, mobile carrier. When the app opens for the first time after install, the MMP generates a corresponding fingerprint ID using the same data.
A long enough list of device-level attributes can theoretically ensure that no two devices get the same fingerprint ID and are thus deterministically distinguishable across platforms. In practice, fingerprinting is unreliable since IP addresses change frequently as the end-user IP address changes based on moving around in the world.
Fingerprinting is fundamentally flawed on iOS14 for several reasons:
- Outputting a deterministic output from non-PII is impossible. There’s no way to predict with 100% certainty that a campaign drove an install using fingerprinting methodologies
- Rounding a probability up to 100% loses important data. When we round a prediction that a user to a campaign to 100% we lose important data about other campaigns that may have driven that install. This revenue should be accounted for if one wants to properly predict campaign performance
- Accuracy rates deteriorate significantly over time using fingerprinting. This article from Kochava states the accuracy of attribution with a click-to-install time of 3–24 hours as ~30%. Fingerprinting is really only viable for click to install times under 10 minutes
- Lastly, and most importantly, fingerprinting has been explicitly restricted by Apple as part of their iOS14 announcement without first gaining permission from the end-user through the ATT popup.
10. Why you can’t use deterministic data to infer campaign performance on iOS14
It may be tempting to use deterministic data from a campaign to infer the performance of the whole campaign or future campaigns.
This approach has two fundamental issues:
- Data from deterministic attribution prior to the launch of iOS14 will quickly become stale once iOS14 has launched and the ecosystem shifts. Using this data for any form of machine learning (supervised) model will result in a model that deteriorates in performance over time.
- To effectively deterministically attribute using IDFA, access to the identifier needs to be granted in both the publisher and the advertiser app. Therefore, if the end-user gives permission for Facebook to access their IDFA but the user doesn’t give permission to access their IDFA within an app that Facebook is promoting, then deterministic attribution cannot be completed.
Furthermore, using this data to derive campaign performance is suboptimal. Users who opt in to share their IDFA are not a random sample of users and therefore the data collected through deterministic attribution will be a biased representation of the campaign as a whole.
11. Why is deterministic attribution still useful on iOS14? Why can’t I just use SKAdnetwork?
Apple provided SKAdnetwork with iOS14 as an attribution solution and therefore, because of how that technology works, all attribution will be probabilistic. However, deterministic attribution still can play a role in understanding the composition of revenue within a campaign remains important. If 20% of users opt-in to share their IDFA on both the publisher and advertiser apps, probabilistic attribution only needs to predict revenue for the remaining 80% of installs within a campaign.
12. Why is accurately predicting LTV more important than ever?
Predicting LTV and specifically predicting user-level LTV has never been more important. User-level LTV predictions have 2 core uses on iOS14 that are fundamental to accurately predicting campaign performance on iOS14:
- Probabilistic Attribution
User-level LTV predictions ensure the appropriate predicted revenue contribution from each install can be allocated to the correct campaign based on the Campaign membership probability. In the example below the user-level LTV pre for App user 1 is $100 and the user had a 10% probability of originating from Campaign 1. In this case, the predicted revenue is $10.
Without a user-level LTV model, it would be impossible to accurately predict future revenue from Campaign 1.
Furthermore, LTV can be updated over time as more behavioral data (revenue, in-app events) is collected. This means the ROAS of the campaign can be updated as the cohort matures. This would be impossible without a user-level LTV model and the campaign ROAS would be static at the time the conversion value was sent.
User-level predicted LTV is essential for effective use of ConversionValue. Using any other value than predicted LTV as the ConversionValue is suboptimal as one only achieves a partial understanding of the current and future performance of the campaign