What is Champion Challenger and How does It Enable Choosing the Right Decision?
A Champion Challenger is a method that allows different approaches to testing operational decisions in production.
No doubt you have heard about A/B testing within a marketing framework. For example, if you want to achieve the maximum open rate for an email campaign, you could create two different subject lines for random distribution to your target audience. Once a recipient opens the email, the system records the performance (i.e., email open action) against its subject line, then you can measure the open rate based on each subject line to see which one is the most successful.
In this respect, Champion Challenger is similar to A/B testing. It enables you to experiment in order to reach a decision through different approaches, using decision logic to examine each result and decide which one is most successful, such as fulfilling pre-defined KPIs.
The Champion Challenger technique enables you to both monitor and measure decision outcomes in real-time, simply by using different variations of the same decision logic, with the overall goal of identifying which variation is most successful.
The relative success of a particular decision may be contingent on the intended outcome. For example, are you interested in the speed of execution or the number of offers accepted by the customer?
Why is Champion Challenger so Important?
When you are making different decisions, it is critical to understand which ones are realizing the best outcomes. This is different than demonstrating the quality of implementation via historical and test data on new models. Unlike Champion Challenger, these do not show the business impact of the model. Consider those decisions made around bundling options or discount provisioning — you need to understand whether a particular offer is more enticing to the customer or not, and this is where Champion Challenger can really make a difference.
How does it work?
Champion Challenger will take two or more decisions that result from processing transactions and randomly distribute requests to each of these implementations, similar to our A/B testing example above.
Importantly, with Champion Challenger these random requests are unbiased, using established algorithms from the fields of mathematics and statistics. Operations staff may then control the distribution of transactions using variant decision logic. For example, you can decide to process the majority of transactions a certain way, leaving the remainder as a ‘challenge’, in order to see whether these decisions can outperform the current ‘champion’. That is the beauty of Champion Challenger — it can optimize your decision-making by challenging your day-to-day operations.
