Paid and Organic Installs: The effect of one on the other
Two Approaches to unlock the Halo Effect in Paid App Growth
By Arun Rawat and Shu Ming Peh
Collaboration within Skyscanner is common, but this is an incidental case where two teams, India market growth and central growth started working on the same issue around the same time, and were trying to get the same conclusion with two different approaches. It was a refreshing experience for us and we certainly learnt a lot while doing it.
What was the issue and how did we approach it?
Quantifying Halo effect or Incremental Ratio (IR henceforth); if it exists, and the implication on return of investment (ROI henceforth).
Before anything else, what is Halo effect?
Halo effect (hereafter we will refer to it as Incremental Ratio or IR) is the effect that tells us how many paid installs will help increase organic install by 1; if incremental ratio is 0.5, there will be 1 organic install for every 2 paid installs acquired. The incremental ratio is largely affected by paid installs (and App Store/Google Play ranking, competitor activity).
The Skyscanner India squad was finding ways to boost (app) acquisition, and in the process tried to debunk the myth of IR with A/B tests. The idea was to be able to create a sustainable and recurrent paid campaign with a higher ROI if a large halo effect was proven true. A couple of A/B tests were run to determine the validity of IR.
In parallel, the central growth team was trying to quantify the IR through a simple predictive model of organic installs, and the impact it has on ROI.
Outcome of the experiments
Thankfully, both of us got to the same conclusion; IR exists, and impacts ROI with the incremental organic installs acquired. The IR varies between markets and platforms, and it can differ for an array of reasons; for example: ranking, number of paid installs and size of the market. Because of the varying IR, the increase in ROI is dependent on the IR derived.
Since the IR is not a fixed ratio but varies with external factors, the resulting ROI and hence, the decision to run a paid campaign sustainably over longer periods of time would not be a definite one but would vary. However, it gives us a benchmark of the average expectation of incremental ROI from a paid app campaign if we were to run one.
Another important find was that the IR on organic installs is almost immediate (on the day itself or the next day) and there is a very slight improvement in installs (and ranking) after the paid campaign has concluded.
What did we learn from all of this?
Collaboration
Collaboration between teams is always valuable, it gives us insights or knowledge on gap that we might have overlooked. In this instance, India providing market context and knowledge, provided much food for thought on the model’s differing variables which are typically chosen or modified by teams. And in return, central team were able to provide inputs on the analysis and the next iteration of experiment.
Validation and feedback loop
We realized that having two approaches running in parallel could provide some benefit, as it serves as a form of validation and basis for a feedback loop to our own conclusion. A generic model for India was created first to determine if the variables were suitable, and how well the prediction fit of organic installs works on India before extrapolating to overall and individual markets. At the same time, India could leverage our model’s value to get the minimum expected IR for their next iteration of experiment.
Simplifying things
This brings us to the next point of simplifying things or finding the right balance of complexity. A simple model doesn’t mean it is not well thought out or insufficient. The first pass or a minimum viable product should always be a simple approach that is easy to determine. If additional sophistication is incorporated, it can be a spiral effect causing the addition of further complexity to no avail or a model that overfits the problem (which will result in poor prediction).
Decision: trade-off?
At the end of all the experiments, and analysis, it became apparent that a much larger data set is required to explore the phenomenon in a more comprehensive fashion. Since the IR would vary by market as well as other external factors, the incremental ROI for some markets might be large enough to justify sustained paid campaigns whereas that might not be the case for other markets.
Grow with us
Fancy working with a great team in one of our global offices across 10 locations? If you see a role you like and interview successfully we’ll offer to relocate you. Take a look at our perks and our Growth Marketing, Design and Engineering roles within our Growth Tribes.
About the Authors
Arun Rawat
Hi, I am a Senior Growth Manager responsible for the India market growth and based out of Skyscanner’s Singapore Office. I work on owned, paid and earned digital channels to solve the problems faced by the Indian traveler.
Peh Shu Ming
Hi, I am Shuming. I am currently a Growth Executive in Singapore’s office, working in the Growth Tribe. I try to solve data related problems using data science techniques or known mathematical models.
