Top Data Science Use Cases in Retail
with a brief introduction of underlying modelling methods
While we can all agree that Big Data and Machine Learning forms the core business strategy of tech giants like Google and Netflix. Data science has made its way through to most industries, with major corporations putting in place data science teams to take advantage of the data they own and turn it into actionable business insights that will end up boosting the top line.
The retail sector is no exception. In this article, I aim to summarise the most popular data science use cases in retail and touch on the modelling techniques and code examples for the budding data scientists.
1. Recommendation Systems
There is a good chance you have come across a “recommended for you” statement before. Maybe this was while shopping on Amazon, catching up on your latest feed on Twitter, or trying to find a movie to watch on Sunday night cosied up on the couch. Retail companies use recommendation systems in an aim to enhance the user experience while driving up revenue.
A recommendation system can be built using
- Collaborative Filtering
- Content-based Filtering
- Hybrid system { a blend of the two approaches above }
For the Collaborative filtering system, the items do not need to have been defined or profiled in order for the system to work. Instead, it uses historical users’ behavioural data like past feedback, review, views etc. to exploit similarities amongst several users or items to fill out the missing ratings and make recommendations.
For example, for a User A, the system can search for the most similar users, and recommend the items liked by them. This is known as user-to-user collaborative filtering.

Another approach is item-to-item collaborative filtering, where the system will find and recommend the most similar items. For example, based on similar reviews as other items bought by the user in the past. This option is more resource-saving, as similar items for each item can be pre-calculated and saved in a key-value data format, ready to quickly surface when a customer adds an item to the basket.
This method is also more interpretable to the customer, as the system can explain why an item is recommended because “you bought item X”.
For collaborative filtering to work, it needs a rich history of data, which implies that new products that have not been viewed or purchased would not get recommended (known as a cold-start issue).
In Content-based filtering, the items have profiles describing their features (i.e. attribute tags) and users have profiles detailing their preference in terms of attributes. Note that the user profile can be made up from two types of information, the user’s preferences (often user-given), and a history of the user’s interaction with the system (like past likes and purchases).
The system would then recommend an item to a user if the two profiles match, or try to recommend items that are similar to those that a user liked in the past using item attribute data, and recommending the best-matching items.

A limitation of content-based filtering is that it could over-specialise and only suggest new items of the same content type. For example, the recommended dress could be too similar to the previously bought dress and may not be useful for the user.
Industry Use Case
Collaborative filtering approaches are adapted commonly in practice, by Amazon on item-to-item filtering, and ASOS on user-to-user filtering.
ASOS also uses the content-based filter for new products of which it has thousands uploaded each week.
“because you bought item …” — Amazon
“users of similar height and weight to you have returned this clothing in size 8 less frequently than other sizes” — ASOS
Implementation Example
Recommendation systems used by companies are usually built by their in-house data science team. You can read more about Shopify’s recommender system here.
More in-depth post on how to build a recommendation system using collaborative filtering in Python with code example here. A shorter example on the Maths here.
2. Market Basket Analysis
Think of your weekly grocery shopping, can you think of any patterns in the items you put into your basket? Most of us cannot. However major retailers definitely think there are lots to be learned about you by looking at what items you buy together.
Supermarkets and large retailers look for any association of items in transactions and use this information for cross-selling, up-selling, and to influences sales promotions and loyalty programs.
Market Basket Analysis is one application of using Association Rules to uncover associations (so-called “rules”) between items bought together in one transaction.
An Illustrated Example
- assume you have data on 100 transactions
- milk was bought in 20 transactions, eggs in 10, milk and eggs were bought in 8 transactions
- say we’re interested in the association rule of { milk } => { eggs }
- Support is the relative frequency that the rule shows up.
Here Support = P(Milk & Eggs) = 8/100 = 0.08. - Confidence is a measure of how often the rule is true.
Here Confidence= P(Eggs|Milk) = Support/P(Milk) = 0.08/0.2 = 0.4 - Lift builds on the confidence but controls for the frequency of the consequent item (i.e. eggs in this case).
Here Lift = Confidence/P(Eggs) = 0.4/0.10 = 4
Value of support helps us identify the rules worth considering for further analysis. Often in practice rules with very low support value are cut as we do not have enough information on the relationship between its items and should not draw a conclusion due to a low amount of data.
With confidence a higher value meaning a stronger relationship between the items. The actual “cut-off” value would depend on the category of products but generally, it is set at higher than 50% if not much higher.
If a rule had a lift of 1, it would imply that the probability of occurrence of the two items is independent of each other. A value of lift less than 1 means that the items are substitutes, having one item in the basket has a negative effect on the presence of the other item. A value of lift greater than 1 signals for a high association between the items.
Implementation Example
Market Basket Analysis can be carried out with a few lines of code in Python or R utilising existing libraries. For example, in Python, the frequent items can be found using theapriori library, then rules can be built with theassociation_rules library. See a coding example here.
An advantage of association analysis is that it is relatively easy to understand and explain to non-technical people. Since it is also an unsupervised learning tool there is little to do in terms of data preparation and engineering. It can be a good start for certain cases of data exploration and can be used to point the way for deeper analysis.
3. Price Optimisation
Price Optimization Models are mathematical programs that calculate how demand varies at different price levels, then combine that data with information on costs and inventory levels to recommend prices that will improve profits.
- Bain & Company
For companies operating in the B2C space, given how core product pricing is to business revenue. The pricing engines used by large companies are often bespoke, built in-house, and fairly complex. Not surprisingly it was also difficult to find any detailed information, as it’s a trade secret no one wants their competitors to get their hands on. However, generally speaking, a pricing optimisation system may include any or multiple of
- segmentation analysis of customers
- competitor pricing analysis
- demand analysis at various pricing levels
- real-time dynamic pricing { example being the use of surge price by Uber }
Implementation Example
I could not find any code examples. Strategy Consultants Bain & Company frequently advise their blue-chip clients on such topic, charging 0.5 to 1m price tag per project. Software companies have also developed price optimization software packages. IBM, Oracle, Intelligence Node just to name a few of such providers.
4. Customer Sentiment Analysis
Analysing over customer sentiment is nothing new in retail. Instead of using customer polls and focus groups, retailers now look to social media and online product reviews for the data they need.
The most common use of Sentiment Analysis is to turn text (reviews, comments from customers) into a class. Depending on the dataset and rational of analysis, Sentiment Classification can be a binary (positive or negative) or a multi-class (positive, neutral, negative) problem.
Implementation Example
- Step 1: pre-processing
- Step 2: classification
An initial step in sentiment classification is pre-processing where various techniques are applied to data in order to reduce the noise of text and reduce dimensionality, this is to improve the efficiency of the classification later. For example, remove punctuation, remove numbers, remove stopwords like ‘an’, ‘the’, ‘be’ etc.
Next, to carry out the classification, various Machine Learning approaches can be used here.

