Driving user growth with performance improvements

Sam Meder, Vadim Antonov & Jeff Chang | Pinterest engineers, Growth

In early 2015 Pinterest engineers ran an experiment that improved mobile web home landing page performance by 60 percent and mobile signup conversion rate by 40 percent. However, the experiment was a hacky solution that used a lot of shortcuts like serving pre-rendered HTML pages without using any internal template rendering engines or common resources (JS, CSS). To productionize learnings from this experiment, the entire front end engine, all page templates and common elements had to be rewritten. It was a huge effort, and to achieve it, we needed to start from building robust metrics to track our progress for all parts of the serving system. In this post, we’ll cover how we improved performance on Pinterest pages, and how it led to the biggest increase in user acquisition of 2016.

Measurement

  • navigationStart event is initiated by a user clicking a link or hitting enter after typing a URL in a browser navigation bar.
  • domComplete event is fired when all of the processing on the page is complete and all resources (e.g. images, CSS, JS) have finished downloading. A user would stop seeing the browser tab spinner spinning and additional logic on the page (e.g. onLoad javascript) would be executed afterwards.
Image for post
Image for post
Copyright © 17 December 2012 World Wide Web Consortium, (MIT, ERCIM, Keio, Beihang)

This metric is a good start but has a major drawback: it doesn’t reflect performance. This matters to the user since the visible part of the page could be loaded significantly faster than the entire page.

User perceived wait time

  • domLoading event is fired when the browser receives the entire document and starts to render it.
Image for post
Image for post
Copyright © 17 December 2012 World Wide Web Consortium, (MIT, ERCIM, Keio, Beihang)

As an additional benefit, a similar metric could be introduced to mobile applications and measured in the same way.

We’ve integrated both metrics (overall PLT and UPWT) as well as some extra performance metrics (e.g. server side performance and more detailed browser side performance) into key company dashboards and our experimentation framework. This enables us to track progress and quickly understand which improvements lead to bigger gains.

Learning: Create the right metric to track your progress and focus on big impact changes first.

Optimizations

Frontend

Page weight (CSS/JS/Images/HTML)

Rendering (React)

Early flush/chunked transfer encoding

Transport

Backend

Parallelize as much as possible

Only return what’s needed

Cache where possible

Maximizing growth gains with performance improvements

In order to maximize the overall page improvement, it’s important to be so careful that even small conversion features are reimplemented. Our original pages had a lot of these features, and as we continued to find discrepancies and fix them, our conversion rate continued to go up. The big learning here is to segment pages by page type and web/mobile web to better understand where gains come from and to detect issues with any particular segment. These issues might be masked when looking at the overall aggregated conversion rate change.

Conversions feature checklist

  • Navigation mechanics (Popup? New tab?)
  • Signup and form mechanics (Field validation messages, same fields and steps)
  • Auto-authentication features
  • Mobile web and tablet app app upsells
  • Mobile web deeplinking

Another important thing to do with a performance rewrite is run an SEO experiment on every page type. For more information about the basics of SEO experiments, check out our previous post, Demystifying SEO with experiments. SEO experiments show if page load time improvements actually result in more traffic from search engines, and in our case, it showed that it did. If your page is a highly trafficked page, chances are you also have implemented a bunch of features that improve search engine ranking. An SEO experiment will also show if some features weren’t properly reimplemented. Even small details like image sizes or the HTML tags used can matter, so it’s important to monitor this for all page types. For us, it took a few weeks of identifying and fixing discrepancies to get our SEO traffic on par.

SEO Feature Checklist

  • Identical image sizes
  • Descriptive text
  • Amount of content on first page load

Key Learnings

  • Segment the experiment into different page types and segment web and mobile web
  • Run an SEO experiment also
  • Look into each segment to see if there are any features that are missing that may be decreasing the conversion rate or SEO
Image for post
Image for post
Fixing a small conversion feature brought one conversion metric to par

Results & future

References

How we doubled sign ups from Pin landing pages

How we increased active Pinners with one simple trick

How to acquire mobile users through growth engineering

How holdout groups drive sustainable growth

Demystifying SEO with experiments

Pinterest Engineering Blog

Inventive engineers building the first visual discovery…

Pinterest Engineering

Written by

https://medium.com/pinterest-engineering | Inventive engineers building the first visual discovery engine | https://careers.pinterest.com/

Pinterest Engineering Blog

Inventive engineers building the first visual discovery engine, 200 billion ideas and counting.

Pinterest Engineering

Written by

https://medium.com/pinterest-engineering | Inventive engineers building the first visual discovery engine | https://careers.pinterest.com/

Pinterest Engineering Blog

Inventive engineers building the first visual discovery engine, 200 billion ideas and counting.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store