How to predict the purchase intent of users in e-commerce marketplaces
Find out how we use machine learning to target marketplace users with advertising campaigns based on their specific interests.
An introduction to the Adevinta marketplaces project
The idea is simple: at Adevinta, we want to target audiences with advertising campaigns based on their specific interests. However, in order to do this, we need to identify our users’ interests. As these interests may be seasonal, we refer to them as hobbies. We assume that we can learn about our users’ hobbies based on their previous browsing behaviour. The solution is intuitive: we begin by segmenting users based on their past browsing behaviour, then target and enrich their experiences depending on which segment they’re in.
However, a single user may have multiple hobbies, the hobby may be seasonal, or the interest may be deep in the user’s behaviour timeline. Therefore, segmenting users for advertising campaigns may not result in sales. It may also be ineffective in increasing traffic (visits, clicks, or conversions) to our marketplaces (the platforms where buyers can sell their goods or services to a select group of customers).
Therefore, we redefined the problem to binary classification, where we simply classify whether the user is a potential buyer or not. With this approach, we can find potential buyers in each category and increase sales by targeting advertising specific to them.
What we set out to achieve and why
Our marketplaces have a large user base. Some users are needs-based, one-time users, whilst others are more regular due to their habits or season. HomeNGarden users, for example, are more likely to visit our marketplaces in the spring and summer to improve the appearance of their gardens when the weather allows them to spend more time there. Another example is summer sports enthusiasts, who are likely to visit marketplaces during summer or before, to purchase summer sports equipment. As a result, understanding specific user interests in our marketplaces could be beneficial to reach the right customer, at the right time, with the right product. We call this the power of the three Rs.
Why do we need to reach the right users with the right product at the right time? The most important reasons are:
- To increase user activity in our marketplaces and drive more business
- To motivate customers to stick to their routines
- To find new potential customers
How to hack that hobby
We began working on this seed idea as a team during a hackathon in March 2022. The difficult task was defining the problem. We considered segmenting user behaviours, but that didn’t align with our goals because a single user may have multiple hobbies (overlapping interests), and user interests may change based on the season. We were also unable to learn the segments due to a lack of definitive segment labels. Without clear evidence of whether this solution could generate more traffic and sales, we decided to focus on understanding the purchase intention of individual customers based on their behaviour.
We started by focusing on a single hobby, HomeNGarden, and defined a baseline.
Baseline: customers who purchased in the previous nine months will make another purchase in the coming months.
The baseline results are adequate, with a precision (percentage of correct users caught by the model) of 75% and a recall (percentage of total right users who did the model capture) of 44%.
The initial machine learning (linear approach) models could not outperform the baseline metrics, so we continued with carrying out A/B testing on the email campaign using the baseline results. The email campaign was a success. We set new benchmarks with an 84% open rate of emails and a click-through rate of 0.68%. The baseline model showed a significant, incremental impact on replies and repliers. This demonstrated that targeting potential buyers will increase the reach of our marketplaces, so we reiterated the same model approach.
Dataset preparation
We used first-party data that was readily available including: listing views, searches, searches saved by them, bids, favourites, unfavourites, and messages, if buyers interacted with sellers.
We needed historical purchase information from users for labelling, but peer2peer payments are typically only available for a subset of items in a catalogue, resulting in limited label data to identify patterns from. So, we defined and labelled connection2point (C2P) as a purchase. Below is a description of C2P, with a visual representation in Figure 6. As a result, we were able to generate enough label data for our model.
C2P: A buyer messages a seller, and the seller responds, then the buyer messages back, indicating that both the buyer and the seller made a connection.
Defining the problem for the model to understand
The dataset is a chronological collection of user events, including purchases, but we needed to figure out how to tell the model which specific behaviour indicates purchase intent. The plan was to observe behaviour for a set period and see if it repeats itself. But how precisely and for how long should we examine the behaviour? We needed a mechanism to separate user behaviour into sessions with, and without buys.
We experimented with various static time partitions to observe user behaviour and feed it into a learning model with assorted session-based features. However, these initial partitions were too large to pinpoint potential buyer behaviour.
In the second-hand e-commerce market, the majority of users do not subscribe. Therefore, there is no established contract between users and the marketplace. Users are typically only active for a short period before deciding whether or not to purchase an item, so we chose a more dynamic method of partitioning the behaviour into smaller partitions. We then observed the behaviour in smaller sessions.
We collected user interactions, sorted them by timestamp, and cut them into dynamic sessions. Most of our sessions are 30 minutes or less, following e-commerce industry standards. The following two rules describe how we partitioned the user behaviour dynamically:
- When a p2p (purchase) event occurs
- When there is no activity from the user for more than 30 minutes
The feature engineering stage
The engineering of features is essential as it means we can use data collected from our platforms to make assumptions about our users. For the purpose of this test, we identified and defined the attributes of user behaviour that may represent purchase intent. Based on the research and domain knowledge, we identified the feature set. We discuss a few behavioural attributes and explain why and how they indicate purchase intent.
The number of views
The number of item views in the session: The higher the number of items viewed, the higher the interest in the product or category.Total dwell time
The total amount of time staying in the session: The longer a customer spends, the more interested the user is in a product or category. A positive relationship exists between dwell time and user interest in the items during a session.Step dwell time
The amount of time spent on each product in a session: The higher the level of interest, the longer the user spends on an item.Popularity
The highest and lowest popularity of viewed items in a session: Popularity features indicate how popular a product/listing is in a given session. The more popular the product/listing, the more likely it is to be purchased. It shows that the user has a specific item in mind and is viewing the listing multiple times.Particularity
Whether the user is simply browsing aimlessly (window shopping) or if they are looking for something specific in a particular category: A user who browses in a particular genre is more likely to buy a product from that category.Seasonality
If the user browses in patterns that align with specific seasons: We derive characteristics that represent seasonality because it may be a good indicator when combined with purchasing behaviour.
The modelling stage
We chose a simple feed-forward neural network (Multi-Layer Perceptron) because it is good at detecting non-linear relationships and interactions between features.
The model outperforms the baseline, with a precision of 97% and a recall of 70%. This means the model is good at finding the right customers, but not so good at finding all the right customers with purchase intent.
We scheduled the model in near real-time and derived the likelihood of user intent to buy a product in each category. Currently, potential buyers receive listing offers based on their interests or hobbies but we want to use the results of this model further to customise users’ homepages for better personalisation, in addition to targeted campaigns.
In conclusion — a success!
Our approach worked well with the machine learning method providing stronger results than the rules-based method. With a model that is generic and easily adaptable to other categories or hobbies, we can alter the feature and model setup to find potential customers for specific campaigns. For example, if we wanted to run a campaign to promote sports gear to customers, we could target users who have higher levels of purchase intent in the sports category. Also, we believe that this product has the potential to scale across our different marketplaces.
What comes next?
More improvements are on the way as we do not consider the order of events in our current approach. For example, when a user views the listing, spends significant time on the listing, favourites, and bids on the listing, this sequence of behaviour may indicate a strong indication of intent to buy. Using sequence behaviour and sequence approaches will enable us to improve the relevance and strength of our model. Once the current A/B test results are available, we will keep improving our model to be even more effective in identifying users with intent to buy across our marketplaces.
Do you have any comments on our methodology? Or tips on further improvements? Please get in touch.
