Written by Jose Luis Ricon Fernandez de la Puente — Machine learning engineer at aiden.ai.
Lifetime Value — LTV — is one of the key metrics marketers look at to assess the performance of their advertising campaigns. As a metric, its calculation is in principle simple: Total revenue over unique users acquired through paid advertising. This can be further broken down by cohort to get a perspective on how performance is evolving over time. By comparing LTV to CAC, it’s easy to see if a campaign is paying for its costs.
But in practice, it is never that easy. LTV in principle can keep changing forever. For example, if a user who signed up this month spends in 12 months from now, the LTV for this month’s cohort will change in a year. As waiting that long is impractical for operational purposes, the community of marketing analysts has come up with two ways to work around it.
One is to focus on LTV at different points in time; like LTV after one month, or after 3 months. This way, one can know the value for certain. But, on the other hand, we may be missing revenue that accrues after the threshold. To avoid this, we can choose a very long time period, such as LTV180 (6-month LTV), but then we will have to wait for months before we can estimate the performance of our campaigns!
Another approach is to use approximate formulas, like the well known ARPU/churn rate formula. This in turn has its own issues: it is assuming that, for example, churn rates are constant, which doesn’t have to be true; it is quite common for churn rates to vary over time.
And of course, even for a single account with a known churn pattern, LTV will evolve differently across different segments.
It’s not easy to keep track of LTV across campaigns and channels if one is managing a moderately large marketing effort. Individual modeling of the behaviour of every single LTV curve is possible to do in Excel, but it can eventually get time consuming.
At Aiden.ai, we have decided to tackle this problem with Machine Learning (ML). We believe, as Kate Minogue from Facebook said, that “Model choice is key but that does not mean more complicated always equates to better”; ultimately one should use the tool that works for the task at hand. But in the case of LTV, ML does make a lot of sense: we can simply feed our model with historical data about LTV behaviour, and it will automatically learn how to generate predictions for any sort of marketing assets. We also believe in the importance of models being transparent; and so our model accounts for the uncertainty in the predictions it makes.
At the beginning, predictions will naturally be highly uncertain:
But over time, the model will eventually converge to a much more narrow estimate:
To obtain these results we have used the Gradient Boosted Trees ML model, in particular the LightGBM implementation from Microsoft. Gradient Boosted models are popular in data science competitions (e.g. Kaggle) because of their high performance. LightGBM in particular, when compared to similar models (XGBoost or CatBoost) is faster to train; meaning that we can iterate faster on the model to tune it optimally.
In turn, this tuning was achieved by a combination of cross-validation and bayesian optimization: ML models typically have many settings one can tweak to change the behaviour of the model. Trying every possible combination is impossible, and the tuning step is usually done by hand, with the help of intuition and past experience. Instead, at Aiden, we ran hundreds of model iterations, picking the best next combination of parameters in an automated way, balancing refining combinations that were already seen as successful and exploring totally new combinations. To determine the best combination, our training pipeline splits the data into various chunks, training and testing the model repeatedly to make sure the results obtained are consistent (cross-validation).
In addition to that, in order to produce confidence intervals from such a model once we found the optimal parameter combination, we used bootstrap sampling to train the same model multiple times on diverse subsets of the data, so that at the end a given prediction is actually the prediction from all the submodels. Thus, we end up with a statistical distribution at each point we can derive confidence intervals from.
But our journey doesn’t stop here: We are already working on better and improved models using deep neural networks, in order to further raise the accuracy of the model.
This is great news for marketers! Now, our clients will be able to take action faster than ever before. With Aiden you can, for example, set LTV alerts so that if we expect a campaign won’t reach a given threshold, we will let you know from day one so that you can take action and save money that could be better invested elsewhere.
If you think this article is interesting, please don’t hesitate to recommend it by clicking the 👏 button below. To know more about Aiden: www.aiden.ai
