The Holy Grail of Online Marketing
I just came across this excellent presentation by Prof. Dr. Herman A. Bruno — “How to innovate with data analytics in retail”. It has a very good summary of 5 steps of making the most of your data.
Take a closer look at the step no. 5 — counterfactuals. The Holy Grail question of the marketing.
Short term experiments are usually well mastered by marketing teams. This include controlled experiments, A/B or MVT testing. But the bigger question remains — what is the impact on the company bottom line? Revenue, new customers, items sold. Medium term effect is notoriously difficult to measure.
Medium term impact of the advertising campaign is notoriously difficult to measure. Impact on the revenue, new customers unit sold is one of the key business indicators.
One of the approach to solve it are econometric models. While providing some insight they are usually difficult to implement and master.
One of the alternatives it to approach challenge of measurement and what-if scenarios from different angle.
Going beyond econometric models: Bayesian structural time-series models
This paper proposes to infer causal impact on the basis of a diffusion-regression state-space model that predicts the counterfactual market response that would have occurred had no intervention taken place.
Inferring causal impact using bayesian structural time-series models (PDF).
Implementation in R
An R package for causal inference using Bayesian structural time-series models
By default, the plot contains three panels. The first panel shows the data and a counterfactual prediction for the post-treatment period. The second panel shows the difference between observed data and counterfactual predictions. This is the pointwise causal effect, as estimated by the model. The third panel adds up the pointwise contributions from the second panel, resulting in a plot of the cumulative effect of the intervention.
