Dafang He | Software Engineer, Home Candidate Generation; Andrew Liu; Dhruvil Deven Badani | Software Engineer, Homefeed Ranking; Poorvi Bhargava; Sangmin Shin |Engineering Manager, Home Ranking; Duo Zhang | Engineering Manager, Candidate Generation; and Jay Adams | Software Engineer, Inspire

Intro

Pinterest is a place where users (Pinners) can save and discover content from both web and mobile platforms, and where increasingly Creators can publish native content right to Pinterest. We hold billions of content (Pins) in our corpus and serve personalized recommendations that inspire Pinners to create a life they love. One of the key and most complicated surfaces for…


Jian Wang, Jiaqi Gu, Yi Yang, Isabel Tallam, Lakshmi Narayana Namala, Kapil Bajaj | Real Time Analytics Team

This is a three-part blog series. Click to read part 1 and part 2.

In this blog post series, we are going to discuss Pinterest’s Analytics as a Platform on Druid and share some learnings on using Druid. This is the third of the blog post series, and will discuss learnings on optimizing Druid for real-time use cases.

Learnings on Optimizing Druid for Real Time Use Cases

When we first brought Druid to Pinterest, it was mainly used to serve queries for batch ingested data. Over time, we have been shifting…


Jian Wang, Jiaqi Gu, Yi Yang, Isabel Tallam, Lakshmi Narayana Namala, Kapil Bajaj | Real Time Analytics Team

This series is three parts. To read part 1, click here. Part 3, click here.

In this blog post series, we’ll discuss Pinterest’s Analytics as a Platform on Druid and share some learnings on using Druid. This is the second of the blog post series, and will discuss learnings on optimizing Druid for batch use cases.

Learnings on Optimizing Druid for Batch Use Cases

System Visibility

During the process of onboarding different use cases, we found many critical system metrics were missing in Druid. In response, we added metrics on usage of…


Jian Wang, Jiaqi Gu, Yi Yang, Isabel Tallam, Lakshmi Narayana Namala, Kapil Bajaj | Real Time Analytics Team

This is a three-part blog series. To read part 2, click here. Part 3 click here.

In this blog post series, we’ll discuss Pinterest’s Analytics as a Platform on Druid and share some learnings on using Druid. This is the first of the blog post series with a short history on switching to Druid, system architecture with Druid, and learnings on optimizing host types for Mmap.

A Short History on Switching to Druid

Historically, most of the analytical use cases in Pinterest were powered by Hbase, which was then…


Ryan Cooke | Mánager de ingeniería

This article was originally posted in English; read the English version here.

Pinterest, como muchas empresas de tecnología, depende en gran medida de los datos para orientar la toma de decisiones e impulsar el aprendizaje automático. Esta estrategia depende fundamentalmente de que los datos sean precisos. A lo largo de los años, trabajamos para mejorar los procesos por los cuales corroboramos que podemos confiar en los datos clave para tomar decisiones y asegurarnos de que las métricas sean precisas.

Cómo puede haber errores en los datos

Para aquellos que no están familiarizados con las métricas, este puede parecer un problema extraño…


Bhalchandra Pandit | Software Engineer

Overview

We describe a novel approach we took to improving S3 read throughput and how we used it to improve the efficiency of our production jobs. The results have been very encouraging. A standalone benchmark showed a 12x improvement in S3 read throughput (from 21 MB/s to 269 MB/s). Increased throughput allowed our production jobs to finish sooner. As a result, we saw 22% reduction in vcore-hours, 23% reduction in memory-hours, and similar reduction in run time of a typical production job. Although we are happy with the results, we are exploring additional enhancements in the…


Nishant Roy | Tech Lead, Ads Serving Platform

In May 2020, Pinterest launched a partnership with Shopify that allowed merchants to easily upload their catalogs to the Pinterest platform and create Product Pins and shopping ads. This vastly increased the number of shopping ads in our corpus available for our recommendation engine to choose from, when serving an ad on Pinterest. In order to continue to support this rapid growth, we leveraged a key-value (KV) store and some memory optimizations in Go to scale the size of our ad corpus by 60x. We had three main goals:

  • Simplify scaling our…


Cathy Yang | Software Engineer, Trust & Safety

One of our biggest priorities at Pinterest is keeping Pinners safe, and that includes protecting them from spam. The Trust & Safety team’s goal is not only to catch spam, but to remove it as quickly as possible to minimize Pinner impact.

The goal of spammers is to make money, and the best way to do this is to spam at scale. It’s a numbers game: one million spam emails are much more effective than one spam email. …


Ankita Wagh | Software Engineer, Storage and Caching

HBase is one of the most critical storage backends at Pinterest, powering many of our online traffic storage services like Zen (graph database) and UMS (wide column data store). Although HBase has many advantages like strong consistency at row level in high volume requests, flexible schema, low latency access to data, and Hadoop integration, it doesn’t natively support advanced indexing and querying. Secondary indexing is one of the most demanded features by our clients, but supporting that directly in HBase is quite challenging. Maintaining separate index tables as the number of indexes…


Felix Fang | Software Engineer, Advertiser Solutions Group

Chi Xu | Software Engineer, Advertiser Solutions Group

Pinterest is where people go to plan and shop, making ideas and ads from brands helpful in taking Pinners from inspiration to action. It’s our goal to ensure ads continue to be additive and not intrusive on Pinterest. Because of the unique and powerful first party signals on the platform, advertisers can reach Pinners based on their interests, intent and engagement on the platform.

To help in delivering the right ads to the right Pinners in an audience of hundreds of millions of people…

Pinterest Engineering

https://medium.com/pinterest-engineering | Inventive engineers building the first visual discovery engine https://careers.pinterest.com/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store