Stakeholder Sense-making: How a weeklong team synthesis process turned every one into user advocates

My product team was once handed a prototype that we needed to build and launch; ideally yesterday. This feature would live on one of the most visited surfaces of our product.

The only problem? We had no evidence that this feature solved real user problems. And I couldn’t wrap my head around how it fit with our holistic product strategy. To say I was not excited about our ‘move with speed’ ethos would be an understatement.

Fast forward 4 weeks.

We had completed a research study that my product manager described as “the highest ROI research I’ve ever seen”. And by “we”, I mean all of us.

I had designed two things — (1) a diary study (to collect data from users) and (2) a team sense-making process (where my team made sense of that data). All done remotely.

The result? We had reached an outcome that otherwise would have taken us ten weeks down to four. Our designers had developed a design vision recommending a new-and-improved experience, this time based on research. Our engineers were working away at a near-term roadmap of bugs to fix. Perhaps most importantly, our team had a newfound confidence in what we were doing and why.

The research study consisted of 2 weeks of data collection (via a diary study, an unmoderated research method where users respond to a variety of prompts over some period of time) followed by 1 week of team sense-making. By “sense-making”, I am referring to the process by which researchers make meaning out of data and shed light on the “So What?” question for the team. The feature went live to a small percentage of users a few weeks later.

There’s a unique experience that a sense-making process facilitates. It has people feeling like investigators as they sift through raw data.

It has them connecting the thing they are building or the metrics they typically look at to real world human problems.

It invites them to marinate in their curiosity just a little bit longer before rushing to implement the next thing.

It brings every one on a level playing field — whether you’re a senior engineer or a product operations associate, every one has the same job of advocating for users.

It allows them to trust findings when they’ve seen transparently how the data is analyzed, then given a chance to be considered, and finally gets translated to insights.

I saw all these factors at play, and how they created a momentum and confidence that my team carried through beyond the scope of just this one project.

This is the story of how I designed a team sense-making process, and how I motivated my cross-functional (XFN) team to roll up their sleeves with me without sacrificing research quality. This is not about the value of team sense-making; I’m assuming you already believe it to have meaningful benefits. It’s also not about an ideal sense-making process optimizing for rigor; I’m speaking to the practicalities of those working in industries where decisions are made quickly and research impact is assessed based on the ability to influence those very decisions (Note: there certainly are scenarios where advocating for greater rigor and extended research timelines makes sense, but that’s a topic for another time).

My goal is that you, whether you’re a researcher or in a research-adjacent function, will take away key principles to help you design your own flavor of a team sense-making process.

Genesis

So why did I design a team sense-making process?

We needed to build empathy with users, badly.

We needed to do this quickly.

We needed a morale boost, and some way to be inspired to work on a feature that someone else thought was a good idea.

And then there were the benefits to me.

First, it helped me synthesize much faster when I could lean on my team’s observations and perceive what resonated most with them. Second, it pretty much guaranteed research impact and immediate socialization of findings.

And finally, if done well, I knew it would garner significant respect for research. When people wear the shoes of researchers, they begin to understand the craft of making sense of qualitative data. They experience the weight of even one single individual’s response. They discover that there are answers to questions they didn’t know to ask. And they understand it requires skill to put data into structures and to simplify knowledge into frameworks (which requires hard choices about what to exclude).

The Process

I decided to use a human-centered design process, one where the product development process is guided by an understanding of humans at the forefront. This design process highlights two important things. First, it proposes that we decide on the right things to build based on having an understanding of users and their context. Second, it suggests a sense-making process that guides the foundational understanding of the problem space.

IDEO’s 3-stage Design Thinking Process is one of many frameworks IDEO has developed to represent human-centered design. In this framework above, time runs from left to right across the x-axis. Below the line are the more concrete; above the line, the more abstract. The process begins with concrete empirical observations, which piece together into user stories. Patterns identified from a multitude of user stories turn into Themes. I’d add Insights to the height of the curve.

Note: For the rest of this post, I will use the following terminology.

• “Participants” refers to the users that I recruited for the research study.
• “Team” refers to the cross-functional members of our product team that worked on this feature. My team included the functions of product management, design, engineering, data science, product marketing, and product operations.

The Researcher’s Responsibility

Now I want to be clear about one thing: this is not about “democratizing research”.

As a researcher, I was facilitating a learning experience, carefully guiding the entire research process. The research study is my responsibility. Scoping the research? My job. Designing the research activities? My job. Developing the final insights? My job. In other words, I wasn’t willing to skimp on quality.

Principle 1: Facilitate, don’t democratize.

My team did not walk away thinking they were researchers or could be researchers; rather, they left feeling more respectful of the craft of research and appreciative of sharing this experience. It was solely my responsibility to own the design of the research study.

I controlled all interaction with participants (this study was a diary study where participants received daily prompts). My team was not interviewing people, nor designing their own questions and tasks. This is a crucial element since much bias creeps in when non-researchers are having conversations with users, labeling it as “research”.

It is the critical responsibility and specialized craft of the researcher to understand how to translate objectives into research activities, develop tasks that mitigate communication bias, and understand when to stick to a discussion guide vs. pivot toward a new area of inquiry that emerges in an interview.

The Pitch

First things first: I needed to sell the why and influence my team to embark on a sense-making process.

For a process like this to be successful, I would need proper structure and commitment; if people were not invested, the whole thing could crumble apart and I could be left scrambling to fill in holes and end up wishing I had worked alone.

Principle 2: Get buy-in from team leaders first.

I called our team sense-making process “Team Mission Week”.

I first pitched my vision to the product manager, design lead, and engineering manager. I emphasized 3 areas, speaking to the love language of each function: for engineering, I focused on how quickly we’d be able to get to insights if the group joined me. For product management, I focused on the single pointed alignment on user problems we’d get to when we go through a dedicated process of synthesis and discussion. For design, I spoke to the user empathy we’d develop when we gave our full attention to these participants and consumed their videos, photos, and words they logged for us. I’ve found that leaders are gladly willing to support a research vision when it aligns with their goals.

Once I had their buy-in, I shared the vision with the broader team (12 people total). I’ve also found that people want to do work that is rewarding and engaging, so I framed the team project as an exciting journey. I also emphasized that researchers rarely offer these opportunities to teams (which is true).

The decision to join was entirely optional, but I needed to know if they were in or out. I was relieved when every member of the team said “yes”. Whether their commitment was due to curiosity, ownership, or even FOMO, I knew my pitch had been successful. Now it was time for the real work.

Principle 3: Stomp out any doubt.

A model of the flow state according to Csikszenmihalyi

Even if a project holds great potential, it can be intimidating for people to attempt something outside of their comfort zone. There can be concerns about failure or concerns about frustration with the process.

To alleviate their fear, I conveyed my confidence in their capabilities.

I shared that I’d offer clear and easy-to-follow step-by-step guidelines. I previewed their scheduled activities, and I estimated their time commitment. This made the unknown feel more manageable and predictable. And each of these pieces of information helped establish trust and confidence in me as their facilitator.

Team Mission Week Begins

It’s important that people have a clear understanding of what is expected of them (vs. what is optional). I shared an overview of the week with the team, and articulated their responsibilities.

The detailed plan had deep dives for each day with a clear objective, agenda, responsibilities, and documents.

Here’s what the week entailed.

Pre-work

Prior to Team Mission Week, I had prepared participant profiles for the team to preview who they’d be meeting. Every team member was to adopt 2 (or more) participants who they would become advocates for. I also trained them on the basics of the research tool we would all be using. As the Friday before Team Mission Week approached, my team was both eager and apprehensive about beginning their work.

Day 1: Analysis

Each person was responsible for delving into the diary study data of the two participants they had personally selected. Working asynchronously, they completed meticulous notes for each participant using a provided template, as well as a final reflection form also using a provided template. The notes were focused on capturing observations and quotes, while the final reflection offered an opportunity to ponder on key takeaways, identify problems, come up with new product ideas, and raise further questions.

This was also the most nerve-wracking day for me of the whole project. What if my team realized it was too much work and decided to drop out at the last minute? I was relieved to see they had followed through (more on my strategies for motivating them later).

My team members estimated that they had spent anywhere between 1 and 4 hours on this step. The spread can be explained by the range of familiarity with handling qualitative data; our product operations associate was accustomed to reading customer support tickets where frustrated users left long paragraphs of feedback, while our engineers were less experienced in this area.

Day 2: Storytelling

We gathered as a whole group for an immersive storytelling session. One by one, we delved into the stories of each participant, with each team member having an opportunity to share their stories using a storytelling structure I provided as a guide. It was important to me to ensure every one had an opportunity to speak and I made a conscious effort to give the floor to our quieter team members to present their participant stories first. Unlike a typical research share out where the audience is listening to the sole researcher share their expertise, here everyone was involved in both telling (stories) and asking (follow-up questions).

By the end of the session, we had become advocates for all twelve participants. We then closed with a final reflection prompt where I asked every one to note the top three user problems they had heard. This last exercise was an efficient way to drill down to what the team deems are the most important problems to solve, captured when the stories are still fresh in memory.

Day 3: Theme Identification

A smaller group of us gathered to pull out themes from our previous storytelling session. After dedicating ample time to each participant’s story the day before, identifying themes felt effortless. Attendance from the larger group was optional.

Day 4: Synthesis

Now on Day 4, I worked alone to shape our themes into a cohesive structure and uncover the deeper “so what” behind them. This task is a core responsibility of the researcher and is where our skillset shines. We identify what matters, why it matters, and what the group should ultimately take away. At the end of the day, I joined forces with my designers to develop ideation prompts for our final day.

Day 5: Brainstorm

The whole team reunited once more, where I kicked off the meeting by sharing the research insights. They now saw the end product of research synthesis and how we went from seemingly infinite pieces of data down to a small set of insights. Finally, we closed with a brainstorming session on 4 prompts that my designers and I had identified the day before.

Design Considerations

The reason I trusted my team was because I had accounted for what could go wrong, and mitigated those risks through careful planning. Here I’ll share those strategies.

Principle 4: Don’t give up control until you’ve created the proper structure.

Leverage templates to capture empirical data

My team took notes using a set of questions I had provided for them. I encouraged them to note direct quotes vs. inferences they were making themselves. This helped guide them toward where to focus. It also teased out the difference between empirical evidence, inferences, and hypotheses.

Create redundancies

I strategically ensured each participant had at least two “advocates”, which helped me avoid a situation where one team member’s biased interpretations would sway our learnings. In such a scenario, another team member would be able to flag that their notes were very different.

Don’t rush to themes too quickly

Having a storytelling session enables the empirical data to shine. Having the team go one by one from participant to participant is a necessary process by which they become an advocate of just two selected participants to all twelve. They will see how patterns emerge across participants, but also how unique scenarios serve as interesting fodder. The unique observations may not make it to becoming a theme, but they remind us how people are difference, and we never know when something stored in our subconscious can help trigger a creative idea down the line.

Principle 5: Design for team motivation.

My past research on educators and their classroom management techniques had taught me a great deal about motivation. I learned that educators take into account both intrinsic and extrinsic motivation, the former being when people do something for personal fulfillment, and the latter being when people do something for external rewards or to avoid punishment. With these in mind, here were some of my practices.

Give people a sense of agency.

I gave them the choice of which participants they wanted to “adopt”, with the guidance that they choose people who were not too similar to each other. Additionally, they could decide how and when they were going to tackle their analyses. They could get a head start the week before if they wanted to. They could chunk it up over multiple sessions. They could schedule a time to knock these out in a virtual room together. I didn’t care how they got it done as long as they met the deadline.

Generate small wins, early.

It’s valuable within complex projects to consider small touches that can generate excitement and help people move through an otherwise difficult set of tasks. The week before Team Mission week, every one received a surprise box of goodies.

This assortment of snacks was sent to every one’s doorsteps the Friday before Team Mission Week.

I also hosted two “viewing parties” where we gathered to watch some of the early data that had been rolling in. During these parties, I played video clips our participants had submitted of themselves. One of the segments featured these participants’ first time encountering our prototype.

There were many “Oohs” and “Aahs” from my team as they observed reactions ranging from delight to extreme frustration. Nothing gets engineers more excited than seeing people use the thing they’re building. We immediately identified usability issues, and my team started to develop familiarity with the participants and the research tool before it was their turn to dive in.

Create accountability

We had a public document for tracking our work. Team members entered their names in the tracking table next to the name of the participant they selected. They also linked their notes in this table. All of this was in a central place for every one to see, and this motivated people to fulfill their responsibilites.

This is an illustrative example of the sign up table we used. Participant first names were listed in column 1 along with their age, gender, and mobile operating system. A summary of all participant profiles was also provided so team members could glean who these people were. In column 2, team member signed up their actual names next to the participants they had selected (I’ve removed their actual names from the table above and replaced these with their job titles). As team members completed their notes, they would link the document to their names so we could see who had completed their analyses and also easily visit their notes.

Principle 6: Leverage the collective’s expertise.

While researchers become experts in our user problems, we certainly are not experts in how to solve those user problems. We do best to leverage our team for things like coming up with product ideas, defining the scope of brainstorms, and project managing how issues get fixed.

Give people a space for their ideas

As people analyze data, they will naturally come up with ideas and solutions. It’s okay to allow for this even though the research has yet to be completed.

I encouraged team members to chat about usability issues they had discovered in our team Slack channel. This allowed for tactical conversations and solutions to be immediately addressed. I also created a central document for people to add any product ideas they came up with as they reviewed participant data. It didn’t matter how feasible these ideas were; in fact, the ideas themselves weren’t as important as the fact that people had some space to get them out and move forward.

Plan the brainstorm with your product and/or design partner

I collaborated with my designers to come up with our final brainstorm prompts. Although I had a point of view on what these prompts might be, I wasn’t the one who would need to come up with a design vision or product roadmap after this study. So we worked together to shape 4 prompts that blend a few needs: one prompt was fairly blue sky; one was a user interface design challenge where one designer was getting stuck; the last two prompts anchored on significant user challenges we learned from the research.

Principle 7: Don’t forget your job.

And here’s a secret I didn’t share with my team: I still analyzed most of the data, behind the scenes. I had a head start two weeks prior, as soon as the diary study data was trickling in.

There were certain areas I prioritized and made sure to review for every user — the sections that were more foundational (i.e. about the user’s mindset, motivations, and perceived value of the feature) and longitudinal (comparing the user’s perceptions before and after trying out the feature for a few days). These insights would have more lasting impact and could guide the feature’s direction in the long term. I often skipped areas about usability and feature comprehension because I trusted my team would be able to extract relevant patterns on those topics. I focused on being effective rather than comprehensive.

Finale

Here’s what happened as a result of this process.

• By that Friday, low-hanging bugs had already been fixed since engineers had encountered them in their data analysis. My product manager had articulated an emerging roadmap for near-term tactical fixes we needed to make before the feature launched in two weeks. These priorities had emerged during our storytelling session, and were further defined over Slack conversations.
• My design lead had created a new vision for this feature. This vision was peppered with statements like “In research, we found that…” Now, we’d be able to guide decision making for the feature based on user problems vs. business problems.
• I wrote one of my shortest (and fastest) research reports ever. We had opted for team immersion over research reporting. The user stories and pain points had now been seen, felt, and remembered. The report was still important because it allowed me to apply structure and discernment to our team’s data, but I skipped many time-consuming components like narrative development, storytelling, and documenting granular findings. There was so little that depended on this report when most of the impact of this study had already happened.

Here’s what my team had said about Team Mission Week:

“Sometimes our research projects seem to ask our researchers to do a lot of buttoning-things-up or polish at the end with a large report. I think it has its place. But I also appreciated that this study was lightweight. It was totally oriented to impact, and I think the “work to impact” ratio was enormously high as a result”

“The long, focused, in-depth reflections and brainstorms were something I felt like we were really missing from our current team workflow. It seemed like we were cramming super important, tough questions into 30min meetings.”

“It helped the week feel more like an actual team event which is hard to accomplish remotely.”

“This was amazing. Not every research study should or can be a diary study, but I’d love to pull out the pieces of this process that were unique, and apply them to other work we do together.”

All 8 people that filled out a feedback survey about Team Mission Week responded that they’d do this team process again if they were given the opportunity; one person said they’d want to take on even more responsibilities.

Reflecting Forward

While this particular case study might be a unique situation, I believe the principles here can be adapted to any study where researchers decide to bring their teammates into the sense-making process.

As a thought exercise, you might consider modifying a few variables of this study to see how these principles would translate in action: imagine if we had (1) two researchers instead of one, (2) fewer team members who committed, (3) double the number of participants, or (4) the ability to do this in person together rather than virtually.

Stay tuned for posts about different scenarios where I’ve designed stakeholder sense-making processes — i.e. in the context of international field research (vs. a diary study), across multiple product teams (vs. one team), and sometimes over the span of a few months (vs. one week).

Photo by Matteo Vistocco on Unsplash

Special thanks to Altay Sendil, whose thought partnership in the design of Team Mission Week gave me the confidence to go for it. Additional thanks to Velma Velazquez and Tiffany Wexler for being models of collaboration who consistently demonstrate that a united team is greater than the sum of its individual members.

Special Considerations

Designing a stakeholder sense-making process demands a higher intensity workload for a researcher, especially upfront. There were two things in place that convinced me this effort was worth it: (1) this was a high priority and high visibility project, and (2) I had a strong existing relationship with the product manager and design lead, which gave me confidence we could pull it off without too many unforeseen hiccups. Had these two components not been in place, I would likely have taken a different approach or paired down the team involvement considerably.

Since this project, I’ve facilitated many more sense-making processes involving my XFN partners, but not in this “all hands on deck” version because I didn’t think the circumstances were worth the effort.

What about field research?

Team sense-making is valuable in field research contexts as well. However, field research consists of significantly more observational and contextual data than this diary study case study did. It also introduces dynamics between team members and participants that can significantly impact the data being collected (i.e. a user’s comfort level, feelings of respect, and affinity toward the interviewing team).

If I were to bring team members into the field, I’d apply a very critical eye toward selection of who joins and how many as these decisions can impact data quality and logistics. The greater team can still be involved in a sense-making process, but their roles might come later (for example, joining the storytelling session as listeners rather than storytellers).

Is the human-centered design process the best way to develop products?

I’m not sure, but I’ve seen firsthand time and time again the benefits teams derive from starting with a phase of understanding their users. However, sometimes teams don’t have the resources or experience to go through a process like this, especially at startups. They may, instead, start by generating ideas and getting quick prototypes out there to get feedback and iterate as they go. In such cases, the solutions are informed by the perspectives (and biases) of their creators. And often times, the foundational understanding of peoples’ needs happens informally, collected over the span of multiple conversations and iterations.

This certainly can work, though striking success might not happen as often as companies would like to think, as it relies on the team having great intuition and/or luck, and the willingness to pivot quickly when they learn that their initial solution had been in search of an imagined user problem.