Ankit Patel | Software engineer, Content Acquisition and Media Platform
With the growing need for machine learning signals from Pinterest’s huge visual dataset, we decided to take a closer look at our infrastructure that produces and serves these signals. A few parameters we were particularly interested in were signal availability, infra complexity and cost optimization, tech integration, developer velocity, and monitoring. In this post, we will describe our journey from a Lambda architecture to the new real-time signals infrastructure inspired by Kappa architecture.
In order to understand the existing visual signals infrastructure, we need to understand some of the basic content processing systems at Pinterest. Pinterest’s Content Acquisition and Media Platform, formerly known as Video and Image Platform (VIP), is responsible for ingesting, processing, and serving all of Pinterest’s content on every surface of the application. We ingest media at a massive scale every single day. This post will not go into details about the ingestion and serving part, and it will mostly focus on the processing part, as that is where most of the magic happens. …
Philip Apps | Data Scientist, Ads Quality
Clickthrough rate falls prey to optimizing for clickbait and ignores other signals and position bias. A better metric benefits Pinners seeing an ad and the advertiser who created it.
How can we measure user engagement in an online platform? Clickthrough rate (CTR) is the first metric everyone thinks of, but it suffers from some serious shortcomings. Many companies have made tweaks to make this metric more useful — we’ll discuss what we’ve done here at Pinterest.
This post will be the first in a series examining metrics for users, advertisers, and the company — the three stakeholders that online platforms need to satisfy. We’ll focus on Pinterest’s ad marketplace, but the principles can be applied to many other online settings. …
How experimentation and cross-functional collaboration are key to making a redesign successful
Joana Monteiro | Product Designer, Growth & Marie Carter | Software Engineer, Growth.
With more than 442M monthly active users on Pinterest, we must ensure existing Pinners have a great experience, as well as new users who visit the site for the first time. For example, for many, the desktop homepage is a first introduction to the site and gives them an idea of what Pinterest is about and what they can expect to find. For returning users, it’s ideally a familiar feed that represents what they know and love about Pinterest. …
Urvashi Reddy | Software Engineer, Engineering Productivity Team
Adam Berry | Tech Lead, Engineering Productivity Team
Rui Li | Software Engineer, Engineering Productivity Team
The Engineering Productivity team at Pinterest came across a small change that had a large impact in reducing build times across pipelines. We found that setting the refspec option during git fetch reduced our cloning step by 99%.
The Engineering Productivity team at Pinterest is responsible for supporting the engineers who build and deploy software at the company. Our team maintains a number of infrastructure services and often works on large scale efforts — migrating all software at Pinterest to Bazel, creating a Continuous Delivery platform called Hermez, and maintaining the monorepos that get committed to a few hundred times a day, to name a few. …
Deepak Agarwal | Head of Content & Discovery
Changing jobs in the middle of a global pandemic? Stressful. Joining a company to lead Engineering for a product that millions look to for inspiration during said global pandemic — doubly so, but also incredibly rewarding. We sat down with Deepak Agarwal, Pinterest’s new Head of Content & Discovery, to discuss his first three months at Pinterest and his vision for the team’s future.
I joined Yahoo! as an individual contributor when the site’s homepage was the most visited site on the internet. It was entirely hand-curated, though, and our goal was to automate it through machine learning. This work had a tremendous impact on Yahoo’s business, and great technology was created in the process. Back in those days, the Yahoo! front page was the most visited site in online media. It was a firehose that shaped traffic to other downstream Yahoo! pages like Finance and Sports, and to many other external media sites like TV network pages, etc., that were dependent on Yahoo! for leads and traffic. …
James Ouhyoung | Growth Search Traffic, Bruce Yu | Growth Search Traffic
Search Engine Optimization, or SEO, is an important channel for people to find ideas and inspiration on Pinterest, and can also help expose new users to first use cases, such as hairstyle tutorials or a home office setup.
One of the ways we gain a successful SEO presence is by surfacing the best content (among billions of Pinterest pages) via the use of interlinking.
On the Pinterest Growth team, we champion the use of experiments to validate the hypothesis and verify results. After some extensive research, we determined our best path forward was to develop a standard interlinking framework of our own, which we’ve been using to measure the impact of interlinks. …
Rui Huang, Song Cui | Software Engineers, Content Interest Understanding Team, Jennifer Zhao | Software Engineers, Content Core Signal Team, Sai Xiao, Felix Zhou | Software Engineers, Shopping Discovery Team
Pinners have always used Pinterest for shopping, as they often come in a planning mindset and are exploring products and styles. It’s our job to help them go from inspiration to purchase, and so over the years we’ve evolved our shopping products to make it easy to discover products and brands that match individual tastes. As a result, shopping has only grown as a company priority and use case.
Additionally, shopping on Pinterest has increasingly provided retailers with more ways to get their products and brands discovered. We recently launched visual search ads, and more new surfaces for shopping ads, as well as introduced shopping to UK Pinners. …
When I visited the Pinterest HQ to try out my commute in early March, I didn’t realize the trip would be my first and last in 2020.
Cindy Xinyi Zhang | Data Scientist, Experimentation and Metrics Science
On March 2nd, exactly one week before my full-time position at Pinterest started, I went to the company’s headquarters to try out my commute. I jotted down the time my bus arrived and noted exactly when I reached the building to calculate my precise commuting time. After all, you don’t receive the fancy ‘Data Scientist’ title without being at least a little nerdy. …
Nishant Roy | Ads Serving
The ads-serving platform is the highest-scale, highest-complexity, and highest-velocity recommendation service at Pinterest. Our ads business is growing and expanding, and the ads engineering team is iterating quickly to continue to improve the system. Therefore, it is vital to keep the system healthy, in order to protect Pinner experience, business health, and maintain a high developer velocity.
Ever since the first ad was shown on Pinterest, all our ads-serving backend services have been deployed automatically and continuously. Changes are first rolled out to a single machine, which we monitor for very obvious problems like service crashes, or large increases in error logs. Next, we roll out to 1% of the production fleet, where we let the changes rest for two hours, which allows us to detect more nuanced issues like a drop in a certain type of ad, or a large variation in ads from a certain candidate source. It’s very hard to write unit tests to catch these bugs, because the symptoms typically only show up at scale, when the system is serving thousands of requests per second. …
Poorvi Bhargava | Software Engineer, Homefeed Recommendations, Sen Wang | Software Engineer, Homefeed Recommendations, Andrew Liu | Tech Lead, Homefeed Recommendations, Duo Zhang | Engineering Manager, Homefeed Recommendations
The Pinterest corpus is composed of billions of Pins, however, each Pinner only sees a small subset based on their interests when browsing their home feed or other recommendation surfaces. How do we provide these recommendations to each person?
Pixie is one of Pinterest’s major recommendation systems used for fetching relevant Pins. Pixie is composed of a bipartite graph of all Pins and boards on Pinterest. Starting at a Pin that has recently been interacted with, we perform multiple random walks along the graph to generate thousands of similar Pins for that Pinner. These Pins are sorted by “visit count”, or the number of times it was “visited” during the random walks. …