Adopt-A-Participant: Five Tips for Engaging Stakeholders in Large Foundational Projects
How can you meaningfully break down the walls between your research participants and stakeholders? Here, Spotify Senior User Researchers, Akshay Verma and Kathy Lin, dig into the tips and lessons they learned from creatively engaging 40+ stakeholders through a longitudinal diary study with 60+ participants — and how they humanized research along the way.
Let’s face it: stakeholder engagement can be tricky, especially for longer-term, foundational research. And it’s trickier still when teams are working remotely. Despite the privilege of being able to work from home during this pandemic, remote work has added an extra challenge on how to inspire cross-functional partners throughout foundational research.
Last year, we realized that while we are in two different product areas, our teams and stakeholders were asking similar research questions that revolved around the portfolio of Spotify’s personalized playlists. To get everyone on the same page, we decided to conduct one large foundational project together — knowing that working collaboratively would let us be more ambitious with the study and expand the impact of our work. We ended up designing a diary study, enrolling over 60 participants to report on their listening behavior across eight different personalized playlists, with stakeholders from both of our product areas.
This meant that we had a lot of stakeholders and a lot of data — with a long research timeline.
Since the study was going to be so big, we started thinking: how do we make sure that the insights from this study stick with our stakeholders?
Collaboration is a key Spotify value. As is innovation! So we put our heads together and tried to come up with a strategy that could keep our stakeholders engaged throughout such a lengthy and large project, especially as everyone was working from home.
We ultimately crafted an innovative engagement strategy that encouraged stakeholders to directly engage with research long before the final share-out of findings. We wanted our stakeholders to do some of the research work themselves.
More specifically, we had stakeholders “adopt-a-participant”. That means tracking a participant throughout the 3 weeks in real-time as diary entries came in, while not actually interacting directly with participants. We leaned into this metaphor — calling stakeholders “participant parents”.
We would let stakeholders see and interact with the raw data before we had time to analyze it ourselves. In some sense, we needed to not be so precious about our data and to allow our stakeholders to see the messy research process as it happened. While we were aware of the potential risks (e.g. stakeholders overgeneralizing from limited data), we decided that the benefits outweighed the costs, and put in the work to make this approach effective.
We distilled our approach into 5 different components. Even if you are not doing a diary study, or having your stakeholders “adopt” a participant, you can use these principles to increase stakeholder engagement, especially in a remote work environment.
1. Establishing roles
The first step was to clearly establish what everyone was going to do throughout the study. We did not want stakeholders to feel like they needed to do a lot of work, so we made different tiers of participation that stakeholders could sign up for — ranging from those who just wanted to attend the share-outs, to those who would be fully involved in adopting a participant. We strived to be as explicit as possible about how many hours would be required in each of those tiers.
Establishing clear roles and responsibilities right up front made it clear to stakeholders what they were going to do, and what they would get out of it.
2. Keeping communication open
The second component was leaning into open, frequent, and clear communication. Our motto was that there is no such thing as communicating too much.
With so many stakeholders, we could not just rely on a few Slack messages or emails to get everyone across the entire timeline and structure of the project. So, we hosted a kickoff meeting, held frequent check-ins and debriefs throughout the project, and cultivated an active Slack channel for stakeholders to share observations from their participants.
The Slack channel served as a space for both the “participant parents” as well as casual observers to drop interesting observations throughout the data collection process, which promoted discussions as soon as data started rolling in. We helped facilitate conversations and move them along, and also connected the dots between different observations amongst the stakeholders.
Beyond this, we leaned into our insights colleagues to also help with communication. Our data science partners, for example, also “adopted” participants and were key in helping us keep interest and momentum high over the course of the 4-month project.
3. Making it easy to dive in
The more we prepared, the more our stakeholders engaged. Rather than viewing it as busy work or extra work, our stakeholders saw value in this approach and made time during their day to actively participate in our data collection.
We leveraged lots of online tools to make it easy for our stakeholders to dive right into the process. In some ways, being remote and doing everything online makes our data more visible, because we collect everything through video and audio, and our notes could all be shareable.
We gave stakeholders direct access to dscout, our diary study tool, by adding them as collaborators on the project. We made a clear Google spreadsheet with a note-taking template for them to fill out. We held debriefs with interactive brainstorming activities. And our data science partners even created a dashboard so that stakeholders could review diary entries and behavioral data as they were following along, as our participants had given us permission to look up their individual data for the duration of the study.
At this point, you might be wondering: weren’t we afraid that stakeholders would cherry pick insights or falsely generalize their single participant’s learnings to the entire study? We talked constantly about this throughout the project, and our approach to this can be broken out into the next two principles.
4. Leading the witness
While the first three components deal more with how to organize the work, these next two are more about the mindset we as researchers adopted throughout the process. The first part of this is what we called “leading the witness”. While usually this phrase might have a negative connotation to it, we used it to cheekily describe how we thought about balancing having stakeholders look at raw data with ensuring that nobody only saw data that aligned with their biases. It was crucial for our approach to have stakeholders make sense of the raw data themselves. But, we also dropped bread crumbs along the way to ensure that stakeholders did not cherry pick observations that aligned with their views.
This required two stages. First, we worked together early in the analysis phase to highlight the key themes and takeaways that we saw.
Then we organized a “data immersion workshop” (which was inspired by Emily Chu, an Insights Manager at Spotify). During this workshop, groups of stakeholders reviewed raw data that we assembled — videos, quotes, and screenshots that vividly demonstrated key themes — and took notes on an online whiteboard that we created ahead of time.
Through this, stakeholders now had been immersed with participants beyond just the one that they had “parented.”
Stakeholders then went through a lightweight synthesis procedure, first writing out observations, then grouping these observations by theme, and then attempting to come up with answers to the original research questions themselves.
Ultimately, this allowed the stakeholders themselves to see how individual pieces of data aggregate up to the takeaways. Through working with other stakeholders, within a scaffold created by us, the rich conversations that came out of this session primed the stakeholders for the insights that we presented in the final share-out.
5. Embracing the mess
While leading the witness implies that we, to some extent, created structure around the raw data for our stakeholders, another benefit of this approach was actually allowing the stakeholders to embrace the mess of qualitative data in all of its glory.
Each participant in the study had their own unique experiences, feelings and attitudes. As user researchers, we’re usually the ones doing the work of making sense of all these experiences to come up with key takeaways and findings.
It was important in our approach to allow stakeholders to also see this mess, while providing a degree of structure around it. When stakeholders understood the wide variation in lived experiences of real users, they were actually able to contextualize the takeaways better and ultimately build a deeper understanding of users.
In addition, opening up the “black box” of research even led to greater appreciation of our work as researchers. Our stakeholders frequently mentioned in the debriefs that they gained new appreciation of the role of user researchers — while they only had to work with one participant, they were surprised to know we had to do the same, but with 60.
Benefits and limitations
Obviously such an approach is not needed for every single research project. Here are some benefits and limitations that can maybe help other researchers reflect on whether this can fit their use case.
Benefits:
- Stakeholders become your evangelists. During the data immersion workshop, for example, we saw stakeholders sharing their participant experiences with other stakeholders, working together to make sense of the data across participants. The conversation became a forum where stakeholders talked directly to each other about what they observed, and what that meant for their products. This aided in translating insights for their own product teams, as well as across different product teams.
- User Research becomes a long-term, strategic thought partner. During all of the debriefs, we had a column in a note taking document that cataloged future research questions. Many stakeholders were eager to fill it with their thoughts.
- New research questions can emerge more organically. Our hands-on approach planted the seed of future research, which created a virtuous research life cycle — projects spawn other projects, and the ongoing research agenda is driven by both the researcher and stakeholders.
- Insights share-outs can be more interactive and focus on activation. As stakeholders have early previews into the findings, our share-outs were not just focused on communicating the insights to an audience unfamiliar with the conclusions, but rather, conversations about how these insights can actually be applied.
- Research is humanized and de-mystified. Rather than research being a black box process that happens external to the stakeholder, stakeholders can have a more intimate understanding of the research process. Researchers become collaborators, and research participants are not just mere data points.
Limitations:
- Best for larger, foundational projects with complex qualitative data collection. This type of stakeholder engagement is especially suited for diary studies or unmoderated data collection for a foundational study, rather than evaluative or formative research.
- Lots of work and planning to simultaneously manage stakeholders & participants. Luckily, the two of us had the resources and time to work on this complex project together. It is difficult to imagine doing this with fewer than two researchers working together. You’re not only conducting a substantial amount of fieldwork (which itself has many moving parts, as we all know), but also simultaneously managing stakeholder expectations and keeping them engaged and excited.
- Know your stakeholders! Start with those you already know are “research friendly.” While leading the witness help does mitigate the risk of cherry picking insights, such an approach would probably not work as well if stakeholders were not interested in insights to begin with, or were not particularly receptive to qualitative research. Something this involved is probably something you can implement after you’ve already educated your stakeholders about research through other means.
Ultimately, when applied in the right project, we believe that this approach is worth the effort. By opening up the research process and allowing stakeholders to do some of the research work themselves, we humanized the research process, and built empathy, not only towards our users, but also to our discipline.
Credits
Akshay Verma
Insights Manager
—
Kathy Lin
Insights Manager
