How experimentation and cross-functional collaboration are key to making a redesign successful
Joana Monteiro | Product Designer, Growth & Marie Carter | Software Engineer, Growth.
With more than 442M monthly active users on Pinterest, we must ensure existing Pinners have a great experience, as well as new users who visit the site for the first time. For example, for many, the desktop homepage is a first introduction to the site and gives them an idea of what Pinterest is about and what they can expect to find. For returning users, it’s ideally a familiar feed that represents what they know and love about Pinterest. Which is why in June 2020 we revamped the page on Pinterest where new and logged out Pinners land at Pinterest.com.
Why we set out to refresh Pinterest.com
While the signup page gave Pinners a preview of Pinterest content, it mainly provided an efficient way to return to or create an account, and needed updating. For potential new users that come to Pinterest.com, the page lacked a clear value proposition on the ways it could fit into their lives.
Making room for innovation
In the past, the homepage was highly optimized for conversions. Aspects of the page such as the title, subtitle, layout, and background had all been experimented and proved effective at getting Pinners into their account. As Pinterest grows, so does the need to reach new audiences around the world. As we set out to explore how we could innovate on this surface, we also agreed that there was a level of risk in completely changing our homepage. Nevertheless, we knew that in order to continue growing our product, the long-term benefits could potentially outweigh any initial risks. We needed a team to focus on this problem to find a creative and effective solution. As a team focused on conversion and the logged out product experience, the Unauth Product team in the Growth org was a natural fit.
Backed by a number of cross-functional partners including research, we quickly realized we were missing an opportunity to help educate logged out Pinners through our homepage.
A gradual experimentation process
Redesigning the homepage meant not only modernizing an important part of our product but also delivering an inspirational narrative to those visiting with high intent. Our experimentation strategy revolved around presenting both logged out users and potential Pinners with concrete examples of how Pinterest can fit into their daily lives. For this, we presented some of our most popular categories — Food and Drink, Home Decor, and Fashion — as a way to showcase a variety of ideas catering to different interests and audiences.
Phase 1: Gathering data with the old design
We started by testing a few hypotheses on the old homepage. This was crucial in informing the team, since it provided a number of initial data points that helped us make critical decisions as we moved forward. Based on these experiments, we were then able to formulate some initial affirmations:
Homepage visitors have high intent to create or return to an account
The old homepage was focused on Log In and Sign Up, so our first step was to figure out how user conversions were affected by design changes on the homepage. To test this, we simplified the homepage form and added in an additional step for users to choose Log In or Sign Up. We found that while signups decreased, retention increased suggesting that users who dropped off were either low intent or spam.
Adding search to the homepage distracts from logging in and signing up
Searching for an interest is a great introduction to the type of content found on Pinterest, so we tested making search more prominent, and saw how that affected conversions. We added search both as a top bar and to the middle of the page. People did in fact use search when it was featured, and that version had a 1% increase in retention. Search encouraged Pinners to explore, but it took them away from the homepage and distracted them from the primary purpose of the page (signing up or moving on to log in).
Phase 2: Experimenting with the new design
We used these critical learnings to help us move into testing a brand-new approach to homepage.
Content does matter
The old homepage had a variety of sample Pins in the background to demonstrate the types of content found on Pinterest, and we wanted to know how content from specific categories would perform. We ran several experiments to determine the impact of changing Pin content.
Using the old homepage design, we first ran an experiment to switch out the original general background Pins for specific Pin types.
Later, we tried featuring general vs. specific use cases using the new homepage design. After that, we added in an animation to cycle between several use cases on the same page to increase the variety of Pins seen.
With both the new and original designs, we saw Pinners were indeed drawn to the broad visual content. Topics that were too narrow and didn’t appeal to a wider audience, such as women-only fashion, showed drops in conversions. Having a wide variety of content performed better than any one use case, but having a way to show multiple use cases through animation mitigated that difference.
Animations are great! (but tricky to get right)
As seen in the content experiments, we were able to use animations to convey the variety of ideas found on Pinterest. We tested adding animated content below the fold to illustrate Pinterest’s features to understand if Pinners would interact with additional content.
The animation follows a set of Pins as they animate through several distinct transitional and paused states. Each animation was triggered by scroll but the page was not freely scrollable, so we added in some helpful arrows and copy to help Pinners understand how to navigate. One of the biggest challenges was making the animation work for different screen sizes while using animations that required specific values for movement.
To better understand how the new pages performed, we partnered with qualitative research. We learned that Pinners were highly aware of navigation and animation issues such as animation starts on unintentional scroll and no ability to navigate back to the last “page” or paused position. Taking these learnings into account, the final shipped version removed some animations and simplified the page, helping Pinners focus on the core message.
Cross-functional work as the key to a project’s success
The homepage redesign would not have been possible without the cross-functional collaboration that stretched to several parts of the product, including the design, research, content strategy, brand, marketing and international teams. This really proved the effectiveness of having strong cross-functional partnerships and alignment as key to pushing the product forward and turning a vision into reality.
In addition to our findings from experiments and research, we also learned:
- When trying to break out of a highly optimized experience, you need to have clear guidelines and alignment up front for what ‘success means’
- Early insights from multiple teams can help make sure time and resources are well invested before building out full experiences for experimentation
- Longer projects are at risk for scope creep — make sure to clearly delineate MVP requirements and hold yourself accountable to them
“Being able to combine insights from both experiments, at scale, and qualitative research, where we went deep with a smaller group of people, helped build a clear picture of how people were experiencing the redesign. We saw things in the experimental data and we were able to look to the research and say, ‘oh that’s why people are doing that.’ And vice versa, where we heard people express certain sentiments or engage in certain behaviors in qualitative research and could use the experiments to understand those things at scale.” — Chris Schaefbauer, User Experience Researcher
“When it comes to content strategy and UX writing, it’s easy for fast-paced teams to treat these things as an afterthought — ‘It’s just a few words, right?’ What the team saw early was that for this surface to be a success and to tell a clearer and more compelling story about Pinterest, we needed to up our collaboration game. We started with the content narrative and visual model first on this project, and then iterated, iterated and iterated — after each test and looking at data together as a team to deliver a much richer story about Pinterest.” — Matthew Shearon, Content Strategist
A big thank you to all the teams at Pinterest that helped envision and push through months of experimentation and implementation in order to make the new homepage a reality. Finally, a special shoutout to our core team: Michaela Tedore, Shahrouz Tavakoli, Lena Ziskin, Julia Cochran, Brian Lee, Chris Schaefbauer, Matthew Shearon, Ken Saathoff, and Jose Contreras!
- Marie Carter | Software Engineer, Growth
- Joana Monteiro | Product Designer, Growth