Why Your Churn Model Isn’t Working
Instead of focusing on the model itself, focus on what you’re doing with it.
Years ago, I sat in front of a group of executives and proudly presented the work my team had done. More customers than ever were cancelling their services, and my team had been tasked with reversing that trend. After weeks of interviews with business experts — combined with no small amount of number-crunching — my team had developed a machine-learning-driven model that could predict which customers were most likely to cancel their services next. In other words, a “churn prediction” model.
After the presentation, one executive from the finance team approached me. He was very impressed with our work, and asked a seemingly-simple question that stops most predictive churn models in their tracks:
“Your model works on probabilities — even the highest risk customers have some probability of staying. How do we know that we’re not going to spend more money trying to retain these customers than if we had just let them churn?”
The answer lies in choosing the right actions for the right customers.
The business case for a churn model
If you work for a business with repeat customers or recurring revenue, you’ve probably heard of predictive churn modelling. You may even have found yourself in a situation like mine, weighing the benefits of customer loyalty versus the costs of retention. Before we can start to answer that question though, we need to understand the business case for a churn model, which is best explained with an example.
Imagine you are working for a subscription business where each customer pays you $100/month. You develop a churn model that identifies a high-risk group of customers that churn at a rate of 10% per month. Working with your loyalty team, you develop an offer that would give a 5% discount to your customers. You estimate that by giving this offer to the high-risk group, you can reduce their churn rate from 10% to 2%.
If you were to do nothing to that group of high-risk customers to prevent them from leaving, in one month that group would be generating 90% of the revenue they were before. On the other hand, if we offered them a discount, we would retain 98% of those customers at 95% of the revenue since every customer would be paying 5% less (even the ones who would have stayed anyways). If we multiply the 98% of customers by the 95% of revenue they’re paying, we find that we’d retain 93.1% of the overall revenue from that group, meaning that we save 3.1% more revenue by trying to save these customers as opposed to doing nothing.
Unfortunately, the math isn’t as forgiving for many churn models. If the churn rate only decreases from 10% to 9% by offering a 5% discount, then the business only retains 86.45% of revenue by attempting to save these customers. In other words, the cost of trying to save these customers would outweigh the benefits.
The three key variables
When companies run through the above math, the key points of leverage become obvious:
- How accurate is the predictive model at identifying customers with a high risk of churn?
- How expensive are the proactive actions?
- How effective are the proactive actions?
Unfortunately, many companies never bother performing these calculations before deploying churn models, resulting in well-intentioned proactive churn prevention actions causing revenue losses rather than gains. This often causes teams to blame the predictive model for “not being accurate enough,” followed by data science teams using valuable time to try to increase the accuracy of the model by gathering more data, testing new algorithms, or performing additional hyperparameter tuning. While all of these are useful tasks, the accuracy gains tend to be marginal and aren’t enough to turn the business case from negative to positive.
How to improve your business case
Start by taking inventory of the actions you’re taking (or would like to take) based on the output of your churn model. If you’re struggling with ideas, try speaking to your customer loyalty and retention team or reviewing records that explain the reasons why customers have left in the past. Using this list of actions, you should be able to quickly categorize them into high or low cost and impact.
The next step is to perform a series of controlled experiments, focusing on the tactics with low cost and high impact. The purpose of the experiments is to quantify the cost and impact of each action more accurately. Once these experiments are complete, you’ll be able to plot them on a matrix.
The last step in the process is to map actions to outputs from the churn model. The actions in the top right quadrant (high-cost/high impact) should be reserved for the highest risk customers. Customers with a medium level of risk should be treated with the actions in the top left quadrant (low cost/high impact), with the objective being to reduce their level of risk before they become high risk. The lower impact items can generally be ignored, as the resources required to launch and maintain these programs are likely not worth the benefits. All of these actions should be subject to the equations above — if they don’t result in a positive business case, they shouldn’t be deployed.
Conclusion
Thankfully, when the finance executive asked the question, my team and I had done our homework and were able to show how we had mapped the effectiveness and cost of each action we were recommending to the customer’s risk level. The churn model ended up being a resounding success, increasing the company’s overall revenue. By following the guidance above, you may be able to do the same for your company!
Slalom is a global consulting firm that helps people and organizations dream bigger, move faster, and build better tomorrows for all. Learn more and reach out today.
