The “My” in MakeMyTrip: How Per-User Personalization Boosted Conversion at MMT
TL;DR: Using past interaction data, MakeMyTrip personalized hotel ranking for each individual user to get more than 30% upside in conversion.
Hotel ranking tries to solve the hotel discovery problem for users. For a given user at a given time, there is a particular hotel need. For example, the user might be looking for a hotel for the next business trip or an upcoming vacation with her family. The objective is to elicit/infer this need from the user as completely as possible, and then rank hotels in our inventory so that the hotels that best satisfy the user’s need are at the top. This is a non-trivial problem. We found that, for many users, the hotels that they were interested in were not near the top of the hotel listings that they saw. This increases the work the user has to do find a suitable hotel, and increases the chance of the user dropping off.
A user’s hotel need is elicited/inferred using several signals:
- Query (city, adult, children, check-in, check-out, …)
- Device (hardware, OS, browser, …)
- History (bookings, cancellations, reviews, …)
- Recent funnel activity (views, filters/sorts applied, …)
We can measure the satisfaction of the user with respect to the listing shown to them by analysing their click through and purchase behaviour. We assume that the “best” ranking of hotels will order the hotels in decreasing order of the probability that the hotels gets booked. In this article, we discuss an ranking approach using past user interaction data that gave us significant improvement in conversion (>30%).
The first step was to measure the affinity of a user to each hotel. The affinity metric should satisfy the following properties. The metric should be:
- low for hotels where the user has done negative interaction (e.g. given a negative review)
- medium for hotels where the user has not interacted at all.
- high for hotels where the user has had positive interaction (e.g. viewed, booked, given positive reviews etc.).
- more sensitive to frequent interactions than to rare interactions (e.g. a hotel with more views should have a higher value than one with fewer views).
- more sensitive to recent interactions than to older interactions (e.g. a hotel viewed yesterday should have a higher value than a hotel viewed last month).
Constructing an affinity metric that satisfies the above properties is beyond the scope of this article. For ease of exposition, we will consider a simplistic metric for hotel-user affinity that uses past viewing data of users. It is easy to visualize this as a bipartite graph of users and hotels. There is an edge between a user and a hotel if the user viewed the hotel. The weight of the edge is proportional to the number of views of the hotel by the user.
We can compute such a score for every possible hotel-user pair using the weight of the corresponding edge.
In the next step, we use the hotel-user affinity to measure the affinity between two users, in terms of the similarity in their interaction behavior. This affinity metric should satisfy the following properties:
- The metric should be low where there is no intersection between the sets of hotels that the two users interacted with.
- The metric should be high when there are many hotels in the intersection.
- The metric should be high when the hotel-user affinity of both users is high for the hotels in the intersection.
To combine hotel-user affinity to arrive at user-user affinity, we start with a user u and take all hotels where she has non-zero affinity (following the dark edges), and then take all other users u’ who have non-zero affinity with these hotels (following the blue edges). These paths can then be used to compute the affinity of user u with every other user u’.
In this way, we can compute the affinity for every user pair in the graph. Note that it is possible to arrive at similar affinity scores for two user pairs with very different amounts of evidence. Hence it is important to estimate confidence bounds for the estimated affinities.
If there is sufficient data, hotel-user affinity can be used to rank hotels. However, this data is very sparse since any given user will have interacted with a handful of hotels in a few cities. However, we now know other users who are similar to this user and can use their hotel-user affinities to suggest relevant hotels for this user. We need a method that can incorporate both these signals and compute a new hotel-user score which satisfies the following properties:
- A user with a higher affinity to the current user should have a higher influence on the score.
- If a high affinity user has high affinity to a hotel, that hotel should have a higher score for the current user.
To compute the score for a hotel-user pair (h, u), we start with the user u and compute the other users u’ who have affinity with this user (the blues smileys). We then accumulate the affinities of the users u’ for hotel h (the blue edges) after appropriately calibrating them based on the user-user affinities (the dashed edges).
In this way, we can compute a score for every hotel-user pair. This score is used to generate the personalized ranking for every user.
This personalization model went live on the MMT app late last year on a small amount of traffic. Based on initial promising results, we slowly increased the traffic over the next couple of months. Till February, we found a steady upside of more than 30% with respect to the baseline. This has now been mainstreamed, and all domestic hotel ranking involves some amount of personalization whenever data is available.
A fundamental problem in most personalization approaches is one of scale. The MakeMyTrip universe has tens of thousands of hotels and millions of users. This makes pre-computation of per-user hotel ranking intractable. Brute force computation at request serving time is impractical due to latency constraints. Three observations helped us to speed up the computation:
- Hotel-user affinity is tractable, can be precomputed offline, and can be stored in an in-memory cache for fast look-ups. Since it is computed offline, this allows us to use sophisticated methods to accurately estimate hotel-user affinity using a variety of signals.
- User-user affinity computation can be restricted to only those users with at least one common interacted hotel.
- In the score computation, we only consider users with non-zero affinity and only hotels having non-zero affinity to these users.
Based these observations, the optimized implementation was able to deliver personalized hotel ranking well within the latency constraints.
Other Personalization Approaches
The method described above is an example of collaborative filtering using the memory-based approach. We also tried doing model-based collaborative filtering approaches like matrix factorization. The memory-based approach was found to give more relevant results in an internal user study, and was chosen for final deployment. In addition, we are also looking at content-based filtering models based on customer segmentation and hotel attributes for further personalizing the ranking.