Inclusive research at Pinterest: What it means and the path to getting 10% better

By Whitney Paul & Katie Elfering

In 2020, a small team of researchers at Pinterest began to hunt for resources and tools around how to make our research processes and practices more inclusive, thinking our search would surface a comprehensive off-the-shelf resource or two that we could share back and use as a guidepost for our teams. Of course, that was wishful thinking!

Our initial audit of off-the-shelf diversity and inclusion tools for research came up completely empty. Through speaking with our colleagues, we found that individual practitioners had conviction and integrity and followed an ethical and moral compass, however, we did not find an objective framework or process articulation that our team could easily adapt. So, the journey began of crafting one ourselves!

Some important context before we dive into that journey:

What is inclusive research?

Inclusive research is the practice of actively seeking and incorporating relevant, diverse perspectives throughout a research process to move toward outcomes that serve everyone, particularly those who might otherwise be excluded or marginalized. Only when we deliberately pinpoint exclusion can we work towards creative solutions to make things — products, services, experiences — that work for everyone.

Most importantly, context is critical for defining inclusion as the meaning can vary depending on our research questions. For example, beauty research may consider skin tone, whereas food-based research may consider diversity of palates, religious beliefs, race and ethnicity¹.

Why we believe having a framework for inclusive research here at Pinterest is important

As a company with over 480 million monthly active users² worldwide, the lives we can influence are vast and diverse; this includes genders, faiths, orientation, age, location, language, disabilities, cultures and so many more vectors of diversity. Our team at Pinterest has the ability to fundamentally shape the content people find, consider and leverage in their lives via our platform. We thought about the impact we could have on three different groups of stakeholders:

• For our business: Creating an inclusive company, product and marketing is the right thing to do — and it’s also a business imperative. Inclusion is key to staying relevant in the market. Changes happening in society at large must be reflected in our company, product and marketing if we want to maintain a truly customer-centric culture that serves people around the globe.
• For our audiences: At the end of the day, inclusive research, design and marketing impact ALL of our audiences: our users, creators, advertising partners and investors. If we strive to be a platform for everyone, our audiences should see themselves reflected in our marketing, feel that their needs are served by our product and content and feel welcomed and safe in the virtual space we are creating.
• For our employees: Inviting more perspectives at Pinterest into the conversation will lead to better outcomes. By bringing more awareness to inclusion and having a set of shared principles, we can each understand and act on our responsibility as researchers.

How we explored the topic of inclusive research, and what we learned

Essentially, we “researched research,” with fellow researchers often being our target recruit. Conversations with industry experts, internal Pinterest employees and a few existing resources shaped our learnings and principles. While we didn’t find a lot of comprehensive resources specifically relating to inclusion and research, we did lean on a few organizations that had published information and best practices like We All Count, among others.

A few key learnings along the way:

1. Remember that every decision holds power to be inclusive. Researchers often associate inclusion with ensuring a diverse sample or recruit — however, isolating inclusion to one part of the research process undercuts its true power. Inclusion can underpin decisions and designs at any and all steps of the research process, all the way from project initiation to socialization of the findings. In fact, storytelling and the use of inclusive language throughout the entire project life cycle is one of the most essential parts of conducting inclusive research.

In practice this might mean:

• Evaluating research plans early in the process from the lens of different biases to see how those biases are influencing the research.
• Considering qualitative exploration early on to uncover preliminary insights for more inclusive design.
• Partnering with intention: When working with outside vendors, consider who the bid is being sent to — do they have a focus on inclusion, and can they speak to how they address it within research?

2. Act with courage and a learning mindset. Fear of saying or doing the wrong thing often precludes progress when it comes to inclusion, and many researchers expressed that inclusive terminology and practices often feel like they have a steep learning curve. A common sentiment was that researchers felt like “they don’t know what they don’t know.” A learning mindset and the courage to make a mistake and learn from it is an important part of evolving research practices in general, but is especially salient for inclusion.

In practice this might mean:

• Questioning and pushing back on timelines that don’t provide sufficient time to make research inclusive.
• Being transparent on shortcomings and structural issues that may be embedded in the data.
• Getting feedback from other researchers on the scope and design. Identify what your weak spots may be and ask for input on where the design can be pushed to be more inclusive.

3. Be very, very specific when communicating research goals, findings and implications. As we learned from data equity expert Heather Krause from We All Count, “Ambiguous words are where implicit bias lives.” Implicit bias includes attitudes or stereotypes that unconsciously affect our understanding and decisions related to a person or group of people. Particularly in large organizations where we have to gain alignment with many different stakeholders throughout a research project, specificity is the key to making sure we are on the same page about what we are trying to learn, how we conduct research and how we take action on research findings. Recognizing and stating your assumptions and potential biases (and again, being specific!) at the outset of a project is a good practice when communicating your research goals as well.

In practice this might mean:

• Clearly defining where funding and motivation comes from and recognizing how that can bias research design.
• Communicating inclusive research principles to vendors and holding them to the same standards as we would with in-house research.
• Include stories as a complement to quantitative findings in order to better contextualize the lived experience represented by the numbers.

If we can leave you with just one thing to remember about inclusive research, it’s to not let the perfect be the enemy of the good. In many internal research orgs there is a culture of moving fast and being lean — particularly in the world of digital products — which means we’re often left feeling time- and resource-poor. Rather than immediate, radical change, which can feel overwhelming and paralyzing, look for incremental opportunities to make work more inclusive. On your next project, ask yourself, what steps can you take to create a 10% improvement in inclusion? Ten percent is far better than none, and if we learn and evolve each time we go through the process, the cumulative learning will mean inclusive practices are more sustainable and impactful over time.

Putting our Inclusive Research Principles into Practice

It’s one thing to talk about inclusive research and develop principles around it. It’s another thing to ACTUALLY put them into practice. Across the Pinterest Product Research team, we’ve used these principles in countless ways…here is one of the more recent examples.

Much like the development of our hair pattern search, as we set out to explore ways to make Pinterest more inclusive to all body shapes and sizes, we wanted to start from a place of understanding and empathy. After all, “Put Pinners First” is one of our core values. We had heard from users in their feedback and in our data that they weren’t always feeling seen and represented online (not just on Pinterest). So, our task ahead was to answer: How could we understand the problems and needs better to create a space that felt safe, welcoming and like everyone belongs?

As mentioned above, the first step to inclusion is often recognizing and identifying exclusion. We started there — by asking people what they thought about body size representation online (not just on Pinterest), and then exploring that feedback to understand why and how people felt left out or underrepresented. We talked to people of all body sizes, but centered our efforts on those who were and felt the least represented — those in larger bodies and BIPOC people, specifically. We wanted to center our efforts on the margins, because designing for the most marginalized often leads to better design for all.

We wanted to know as much as we could about this space and what people needed to feel included. We asked our users about what body size inclusion means to them, what body type representation looks and feels like online (not just on Pinterest) and what it feels like if (and when) you don’t see yourself represented. We also asked people what work-arounds or tactics they used to see bodies that looked like their own, and found that people have a lot of “hacks” that took a significant amount of time and effort just to see bodies that looked like them.

We validated the user feedback with experts, partnering with the National Association to Advance Fat Acceptance (NAAFA) and plus-size creators to ensure that we heard from as many voices and points of view as possible. And we co-created — every concept we came up with was shared with this community for feedback to ensure that everyone’s voice was heard and needs were met.

It became clear over the course of multiple research studies and conversations with users, creators and experts that the real goal should be true inclusivity — people just wanted to feel seen from the start without having to do a lot of work to see bodies that look like their own. So that’s what we did. After months of research and development, Pinterest launched our body type technology, which increases representation and diversity of body types across women’s fashion and wedding search results and related Pins…without any effort or action required from users. The results are simply more diverse by default.

But the launch of our body type technology wasn’t the end — we went back to our users to confirm that this worked for them. And the feedback was incredible. Not only did people tell us that they finally felt seen on Pinterest and in their search results, but they felt heard in the process of developing this technology. And that’s the real goal of inclusive research — ensuring that people feel seen, heard and included throughout the entire process.

Our Pinterest Inclusive Research Principles were authored by a team of researchers at Pinterest including Chris Schaefbauer, Daron Sharps, Melanie Costello, Sam Tuken and Whitney Paul. Katie Elfering, Chris Schaefbauer, and Jamie Hwang led the most recent inclusive product research efforts detailed above.

¹ Please note that Pinterest’s Advertising Guidelines prohibit targeting of any audience based on race, ethnicity, religious beliefs and sexual orientation, among other things.

² Pinterest Internal Data, Global, Q2 2023