Computing incremental sales at Untie Nots
Welcome to our blog posts on Computing incremental sales at Untie Nots. We have divided the topic into 5 parts, here’s a link to the rest:
Part 1: You’re here
Part 2: Common methods used in the retail industry
Part 3: Handling bias. Intuitive incremental methods
Part 4: Using a more robust approach. A general overview
Part 5: Using a more robust approach with causal inference
Part 1: Setting the scene
The goal
Here at Untie Nots, we design personalised challenges for retail customers. They each have a unique landing page where they are encouraged to spend X euros on a brand in order to earn Y reward. At the start of a month long campaign, a player must choose his challenges and will collect his winnings at the end.
The goal of this solution is twofold: increase the spending on the promoted brand and increase the overall customer loyalty to the retailer.
Correctly measuring the performance of a Customer Relationship Management solution is essential to see whether it works and where it can be improved. This is especially true in retail where the cost of a solution can be expensive, and opportunistic gains happen all the time. For example, some customers will benefit from a 30% discount for a product that they would have bought anyway, the incremental gain of this promotion would be $0.
This series of blog posts looks at the different ways to measure the effect of the campaign on the player’s storewide spending. We will focus on the monetary amount, but you can use these techniques to measure the incremental in number of visits or basket size too.
How a campaign works
Before the start of a campaign, we will split the customers randomly into two groups, the exposed customers and the control group.
The exposed customers will receive all promotional materials, usually an email. The control group will not receive anything and cannot participate in the campaign.
If an exposed customer selects at least one challenge, he is considered as a player.
Available datasets
- For every customer, we know how much he spent, what products he bought, and how much discount he cumulated for each store visit, during the last year
- We also have some personal characteristics for each customer, for example whether they opt-in into a marketing program or whether they are a gold/silver/bronze customer
Major difficulties
Finding the incremental sales might seem trivial at first glance, unfortunately it isn’t very easy to exploit the data directly because of a few issues. We will go over each in more detail as we go over the different techniques to compute the incremental sales.
Low signal to noise ratio
The incremental can be difficult to measure if the increase in player purchases is small relative to the overall sales. For the big stores, this is often the case (but as you know a few percent uplift really matter in grocery retail); and can be exacerbated by outdated marketing communication channels.
Inconsistent spending across a year
The total spending can substantially fluctuate in a year, for example there is often a spike around the end of the year. This is also true at a customer level as some clients will naturally start/stop coming to the store for external reasons.
Incomplete data on customers
To protect privacy and avoid any unwanted bias in the personalisation of challenges, the data on customer characteristics is very limited. This make it harder for us to predict certain behaviour, for example, appetence for digital media makes a customer more likely to engage with our campaign.
Messy data
The bane of all data projects, real world data is not clean. Some systems are a bit old and can yield strange transactions, for example duplicates for a single store. These are hard to capture, thus it is essential to design a solution robust to anomalies in the data.
Notations used throughout the posts
Populations
- Players: customers who participate in the campaign, whether they win or not
- Exposed customers: customers who have received promotional material for the game; including the players
- Non-players: customers who have received promotional material for the game but did not participate in the game
- Control group: customers who have not received promotional material for the game; they are homogenous to the exposed customer
Periods of time
- Campaign: period of time during which a player can play the game
- Pre-campaign: period of time before the campaign used to create a benchmark or to train models
- Validation period: period of time between the campaign and pre-campaign to validate models, it is not always used
Next part
Thank you for reading this introduction, let’s move on to the solutions most used in the retail industry in part 2.
