Dating a hotel? How we created a personalized “Match Score” for accommodation
Just like in the world of dating, choices can become overwhelming, and while there may be many fish in the sea, all you’re looking for is your best match. This is the story of the Match Score, a machine learning based score that enables one-to-one (1:1) personalisation in order to improve the customer experience on Booking.com and make it easier for travellers to find the property that’s just right for them. This story was written by the Match Score Product Manager (Hadas Harush) and Data Scientist (Benjamin Braun).
Our DNA
Like most love stories, the story of the Match Score began with user research which revealed these interesting insights (a.k.a pain points):
- Our customers often feel overwhelmed with information
- Our customers struggle to compare different properties
Hence, in order to address these challenges, the Match Score’s mission was defined as “increasing customers’ confidence in choosing the relevant product for their specific needs.’’
Just like dating apps recommend matches based on common interests, hobbies, age and other preferences, the new Match Score feature matches Booking.com customers with properties they’re considering by showing them a personalised percentage which represents how likely a customer will enjoy the accommodation that they are currently viewing. While “most cases” personalisation often involves creating a persona and developing or optimizing the product for it, this feature is based on 1:1 personalisation. This means that every customer will potentially see a different score for the same accommodation, based on their past bookings, reviews and preferences.
The Match Score feature was first launched to a small percentage of traffic on the desktop and iOS version of Booking.com in August 2019 and can be accessed through “My lists” from the top navigation menu.
How does the magic happen?
A dash of chemistry, a pinch of sweetness, but mostly the score is generated online based on the customer’s previous bookings, reviews and current search parameters. While we train our model for accuracy and set a high bar for quality, the Match Score is currently only available to logged-in customers with at least two bookings in the past five years.
“We don’t grow when things are easy, we grow when we face challenges”.
Here, we’ll share some of the learnings we’ve accumulated in building the new Match Score feature:
1. Build trust
Just like in any relationship, trust is a key element and plays a pivotal role in this feature. During our research expedition, we realized that we’ll need to build our customers’ trust for this feature to truly be successful. Along our journey, we encountered customers who thought that higher scores were sponsored (which is obviously not the case). Other people claimed that we don’t really know them and their current trip context in order to make the prediction for them. We’re constantly pushing ourselves to do better on this end, and we’ll continue until we get it right.
2. Give space
An additional concern some customers raised was around personalisation infringing on their sense of autonomy. This is why we felt it is important for us to guide our customers rather than decide for them. It’s up to the individual user to use or ignore the Match Score. We don’t use any persuasion messages, nor do we filter or sort the options shown. Rather, we chose to give customers the time and space they need to get into this relationship on their own terms.
3. Create a healthy dialogue
Clear, honest communication is yet another key element to any good relationship. Our research revealed that our customers want to have a say in their score. This means that if a couple is looking to book their honeymoon, maybe a hostel which they previously stayed at isn’t relevant as a comparison point and shouldn’t affect the Match Score for this particular search. Hence, we will gradually enable customers to have an impact on the criteria that contributes to their score in order to best suit their needs at a given time.
4. Be transparent and open
Everyone has their story and when assumptions are made, they’re usually rooted in generalisations. We wanted to be upfront and transparent about what we’re made of and for that to happen, we invested a lot of time in explaining the Match Score to our customers (see explanation screen image below). But we know there are things we can improve, and for us to do that, we need to be open to feedback. As such, we actively invite customers to leave their feedback and comments when using the feature.
A technical sneak peek
The concept of the Match Score is not vastly different from other recommendation systems. Where it does differ, is that instead of sorting items based on behind-the-scenes calculations, we actually present the score to the customer.
The Match Score is interactive, which means that every time the search parameters change (such as the number of guests or length of trip), it goes through an online recalculation. This is done by using factorization machines that take into account both the historical data and the customer’s trip context.
Factorization machines transfer items and users into a latent vectors space. It can also vectorize any side feature we choose, like the search parameters mentioned earlier. The most beneficial part of this transformation is that, while the dimensions of the problem are reduced greatly, most of the information is still kept and expressed in the vectors.
The beauty of factorization-based methods is that we do not need to explicitly state these features for the model to make use of them. Using only the interactions between users, accommodation options and the trip context, the model understands on its own that a hostel is better suited to a solo traveller than to a family, or that an airport hotel is well suited for a one-night business trip.
While using the factorization machines is an interesting approach, a big caveat is the difficulty to incorporate customer feedback. Everything is based on user and item interactions, and the entire dataset must be retrained at the same time, which can be computationally heavy. Other types of models, including deep learning and tree-based approaches, might help us in that aspect.
What’s next?
During the next few months, we’ll continue to develop our algorithm which will get smarter with this score over time, based on each customer’s history and the feedback we will receive.
We’ll also invest in enabling customers to impact their scores and recalculate the score based on this feedback. Stay tuned to see how this love story unfolds…
