Building a universal search system for Pinterest

Zhongxian Chen | Pinterest Tech Lead, Search

On Pinterest, the best answer goes beyond text, but can even go beyond an image too. When you’re looking for eye makeup ideas for example, a video personalized to your skin tone range teaching you how to create a certain look is likely more useful than a Pin showing what the end result looks like. If you’re sprucing up your backyard and looking for lighting ideas, an expert on backyard lighting to follow can be a one-stop shop for information, instead of having to browse multiple sites.

As the types of content on Pinterest grow, our search results must adapt as well. Pinners can use filters to narrow down to the content type they want, but we can make the experience even better by predicting the most relevant answers for them. As a solution, we’re building a system to deliver content from various verticals within one integrated search results page. Today when you search you’ll already see these different types of formats (Video, Shopping, Pinners to follow). Over time these results will become even more personalized and relevant through advancements in machine learning ranking,

Architecture Overview

There are three major components in this system:

  • The query understanding component is responsible for detecting vertical intent based on query tokens as well as historical engagement. It also populates pre-retrieval features that are used in the blending component.

The query understanding component lives inside Anticlimax, a service developed to do query understanding and rewriting. Both the query planning and blending components live inside Asterix, which serves as the root of our search system and talks to all verticals.

Detailed Design

Query understanding and planning components are relatively straightforward to understand, so we’ll spend more time with the blending component.

Query Understanding Component

This component decides from which verticals to fetch results. Since we have user info and the query as inputs from Asterix, we can check which verticals the user has seen recently for similar queries and what they engaged with for those queries. If they did not engage with certain verticals, we will not show more of that vertical in the near future. For all verticals that have not been fatigued after this step, we then run the query through the intent detector for each vertical. Each intent detector returns the intent (if any) with pre-retrieval features that are used in blending. Pre-retrieval features represent how strong the intent is and are used later to blend results from different verticals.

Query Planning Component

This component composes and sends fanned-out requests to corresponding verticals based on detected vertical intent(s). One goal of this component is to trigger a vertical results retrieval if and only if the vertical results will be shown to the user after blending so we don’t waste resources retrieving verticals that are not useful. For a small percentage of traffic, we want to retrieve random verticals instead of the one(s) with predicted intent. The logging from this random traffic is used to produce unbiased training data for the vertical triggering and blending models. If we use production traffic, the resulting model can bias towards the existing model, and verticals without previously detected intent (that previously contained low quality content) will never have a chance to be shown.

We could alternatively pick a small percentage of random users for whom to show randomized vertical results. However, we don’t want any users to always have randomized results since that’s not a good experience, so we decided to use the random traffic approach instead.

Blending Architecture

The above graph shows the components used in blending and how they connect to each other:

  • Post-retrieval features extractor: This component extracts features from all retrieved verticals and main Pin results. We want to use the most important features representing quality and relevance of the results. For main Pin results, we will use relevance score, navboost features and popularity features. For verticals, the features are different depending on the type of content. Since most vertical and main Pin results contain more than one result, we take the minimum, maximum and average value of the result feature to be the feature representing the whole vertical. We only use the top N main Pins in feature extraction, which will be explained in the model training section below.

Blending Model Training

Features are logged in Asterix, and we extract labels from user engagement logs. We use online engagement data to generate labels because it is more scalable compared to offline data. In addition, it can be hard for human evaluators to compare the relevance of main Pin results to vertical results. Features and labels are then joined to create training data. One model is trained for each vertical. Here are some details about the model training:

  • Feature vector: (Q, Pq, Vq), where Q is pre-retrieval features, Pq is post-retrieval features of top main Pins, and Vq is post-retrieval features of the vertical.

What’s next?

We’re currently working on iterations to build and improve our universal search system. At the same time, we’re also working on improving the quality of results from each vertical. As we start to show more and more types of content to Pinners, our eventual goal is to make this system into a platform where other teams can easily plug new types of content into our search results.

Acknowledgments: Boris Lin, Jason Lin, Jiawei Kuang, Junwei Shi, Long Cheng, Lulu Cheng, Rajat Raina, Randall Keller, Yanis Markin, Yixue Li

Pinterest Engineering Blog

Inventive engineers building the first visual discovery engine, 200 billion ideas and counting.