Lessons in growth engineering: How we doubled sign ups from Pin landing pages
Jeff Chang | Pinterest engineer, Growth
A popular topic within growth hacking circles is improving conversion rates on landing pages. Everyone has seen those “10 tips to triple your conversion rate” articles that are littered with general tips (e.g. increase CTA size) and promise gains, which are usually small at best. Instead of trying those general tactics, we doubled page conversions and increased SEO traffic by doing one thing: leveraging data to better understand potential Pinners.
Improving Pin landing pages
The first step to improve landing page conversions was selecting the right page to work on. While the “Pin page” (a landing page for clicks to a Pin from another site) is one of our highest trafficked pages, it converted worse than other landing pages, so we invested more resources into it. At first, we didn’t have much data about which parts of the page were effective at convincing a new user to sign up, so we tried a simpler, more visual page layout.
After testing this new design in an A/B experiment, we learned it didn’t increase signups compared to the previous version (i.e. the control). It also was hard to extract learnings from this design because it was so different from any previous version. Was it because we replaced the board on the right with Related Pins? Was it because we didn’t show as much content after scrolling? In this case, we learned that by taking smaller steps, we could learn more from each new version.
So, we tried a new version more similar to the control, where we allowed the Pinner to swipe through a carousel of Related Pins at the top of the page.
This version also underperformed, but only slightly. The data showed few people clicked on Related Pins, possibly because they were small and difficult to distinguish.
Next, we tried making Related Pins bigger and added attribution so they looked more like regular Pins.
This was a success! We saw a lot of engagement with the which led to more signups. Our hypothesis was that this version better because it illustrated the related content on Pinterest and, in turn, showed the value of signing up. We shipped this version, and it became the control in future experiments.
However, we wanted to see if we could do even better at converting Pinners on Pin pages. Because Related Pins seemed enticing to new users, we wanted to further highlight them by adding them to the normally blank spaces on the left and right sides of a Pin.
We were surprised to find this version performed the same as the control. For our next iteration, we tried something simpler, where Related Pins were only on the right of the Pin.
We were excited to learn this version beat the new control. But, we wanted to do even better. We looked into the user action event-tracking funnels and found those who clicked through on the main Pin (and thus went to the external site) barely converted, but those who clicked on a Related Pin (and landed on the closeup for that Pin) converted at a much higher rate. So, we reduced the size of the main Pin to be the same as the Related Pins and gave the Related Pins grid more real estate on the page.
This iteration was a huge success and beat the previous control by over 25 percent (and that’s compounded on top of the gains of the previous versions!). Compared to our first Pin page, this iteration converted at twice the rate. Our first instinct was to ship this immediately, but instead we looked into the SEO experiment we ran alongside it and noticed that it dropped traffic by 10 percent. (Related post: SEO experiment framework.) If we shipped this Pin page, we’d get a net win (increased signups outweighed traffic losses), but we wanted to do better.
Conversions and SEO
When working on conversions for any page that gets a significant amount of traffic from search engines, you must consider SEO effects. For example, if an experiment increased signups by 20 percent but dropped traffic by 50 percent, the result is a net signup loss.
For this experiment, we segmented the traffic by various verticals, such as web traffic, image traffic and traffic by subdomain, and saw the biggest traffic drop in image search. We compared the images in the two designs, and found the big difference was we shrunk the size of the image. From previous experiments, we know when we change the images on the page, even just the size, image search traffic is impacted since search engines have to recrawl billions of our pages. We ran another SEO experiment where we used the same large-size image file as before, but sized it down to fit inside the smaller Pin.
This change increased the traffic difference from -10 percent to +10 percent, even though the design looks the same visually. Not only did this new layout increase conversions, it also increased traffic to the page. These effects multiply with each other to create a larger net signup gain.
By iterating quickly and thoughtfully, we were able to double Pin page conversions and increase SEO traffic. Here are the key lessons we learned along the way:
- Learn more about users by analyzing the event tracking funnel data from experiments.
- Use past experiment learnings to drive new iterations instead of trying “random” ideas. It’s best to have a hypothesis backed by data for why each new design will perform better.
- The faster you iterate, the faster you learn and see gains.
- If you’re working on converting a page that gets a significant amount of traffic from search engines, running an SEO experiment in conjunction with a conversion experiment is a must. Even if you increase conversions, you might also see a traffic loss resulting in an overall net signup loss.
If you’re interested in growth engineering and love experimenting, join our team!
Acknowledgements: These projects were a joint effort between engineering, PM and design on the User Acquisition team.