Applying Deep Learning To Airbnb Search
Check out our paper on the topic, just published in arXiv
Searching for homes is the primary mechanism guests use to find the place they want to book. As the core of how the Airbnb marketplace functions, the goal of search ranking is to find guests the best possible options while rewarding the most deserving hosts. Ranking at Airbnb is a quest to understand the needs of the guests and the quality of the hosts to strike the best match possible.
Several factors make search ranking at Airbnb a challenge unlike any. To begin with, there are the standard challenges associated with scale. Spread over more than 191 countries and 81,000 cities, Airbnb listings are in every corner of the planet and the definition of the best deal has a lot of local color to it. Adapting ranking to perform well across such a diverse range of markets stretches the limits of optimization.
To make things more complex, each one of these listings is a unique offering. The categories of homes include the familiar as well as the novel, ranging from apartments to igloos. Even within a standard category such as apartments, when different variations along price, location, amenities, decor etc are factored in, how to order the homes objectively is a difficult problem even for humans, let alone solving it using machines.
Presumably all such challenges can be overcome by accumulating sufficient amounts of data. But the Airbnb inventory has an additional twist. Each listing can be booked only once for a night. As soon as the listing is booked, it becomes unavailable for any subsequent search for that night, so no further data accrues for it. This makes the data per listing very sparse. The plot below shows the normalized distribution of impressions per listing, and the long tail nature of the data available.
Consequently, the solution to the ranking problem demands broad generalization from a few examples, memorizing the top results in this case is of little use.
The task of deciphering guest preference has an equally fascinating side. We get to see the guests when they are planning their trips, which is not everyday for most. But once they arrive, the guests engage heavily with search and usually have very specific preferences and budgets. One guest may end up booking a bargain private room, while another a lavish penthouse, but both may start by searching for a place for 2 in Rome. To effectively personalize the search experience, we need to process a huge volume of data in realtime.
The large number of dimensions and the relative sparsity of data on both guest and host side creates the perfect storm where quite a few of the known optimization techniques break down.
So how do we address all these challenges? The tl;dr version of the answer is: neural networks. But how we reached there and how we continue to evolve is a tale with twists and turns. Celebrating the culture of sharing, in a first ever in-depth look at Airbnb search ranking, we have published a paper that describes our journey to deep neural networks.
Access the Paper
Applying Deep Learning To Airbnb Search, pdf 8 pages, published in arXiv, Oct 2018.
We hope the story of our setbacks and triumphs provides insights useful to other teams facing similar challenges. We always welcome ideas from our readers. For those interested in hands on action, please check out the open positions for engineers and data scientists on the search team.
