Amplifying Research Impact

Give your research a voice that can drive change across your company.

In this post, Emily Chu — Insights Manager at Spotify, encourages you to examine how you think about impact, and identify new ways to drive meaningful change throughout your research process. Amplifying impact is akin to a ripple effect — making small changes in how you think and do research can gradually create larger waves of influence.

When we think about the impact of our research, we’re really thinking about the actions and decisions that happen as a result of research. In an ideal scenario, research has direct implications on what we do next. We research behaviors in order to do right by the people using our product and engaging with our brand, we enhance the user experience, and continue to drive key business decisions.

However, it’s not always easy. The road to impact is not simply going from A to B.

Putting together a research report, and presenting it to stakeholders alone does not generate maximum impact. There are other layers within the research process to focus on so that our insights and learnings can affect decisions across our teams and companies.

So, what prevents our research from having more impact?

1. We focus on one type of impact
2. Impact lives in one phase of research

These problems are interconnected. We are going to dive into each, reframe them as opportunities and discuss ways to incorporate these opportunities into your research workflow.

Problem Space 1: We focus on one type of impact

Have you ever wanted to kick-off a study that feels like the right topic to explore, but may be out-of-scope for you to take on? Maybe a study that just feels misaligned with the responsibilities of the team? Where research sits in the business dictates the type of projects, and impact you prioritize. The impact we focus on most often reflects a company’s organizational structure. These silos act as blinders, which prevent us from looking at the bigger picture, and having a greater impact on the spaces that are most important to change.

Embedded researchers sit in a cross-functional team made up of product, design, engineering, and insights. Centralized researchers tend to focus on more ambiguous spaces that have impact on multiple teams, products and features, and inform leadership’s strategic vision. In earlier phases of a growing company or start-up, you may find yourself as the only user researcher or advocate for user research. Your job in this setting is focused on educating and helping your team understand people, and getting buy-in for user research.

If we look at research impact across the organization structures of research, they can be grouped into 3 areas:

• Product and Design Impact e.g. embedded researchers
• Strategic Impact e.g. centralized researchers
• Perception Impact e.g. newly formed research orgs

The impact we want to achieve should not be married to where research sits in the organization. We need to think beyond a single type of impact.

Product and Design Impact — How can we think beyond iterating on a feature?

In this situation, it is challenging to apply our expertise and knowledge to affect decisions around the business and the long-term product roadmap. This challenge exists because of the expectations that our stakeholders have for the work we need to do. Our approaches end up being too limited in scope, like only focusing on tweaking UX/UI to optimize usability. Evaluative research is important, however, if we only focus on this type of work we are unable to get ahead of the current roadmap, and look beyond to inform the future direction of the team.

We need to reflect on what we know, and if there needs to be deeper exploration into new opportunities that may yield a huge impact. Impact in this case may be the holistic user experience (bringing more delight), or the business goal and metrics (helping us achieve our company’s mission).

For example, at Spotify when we were redesigning the free tier of the mobile app, we ran a handful of usability tests to see how we can iteratively get people to start listening right away while delivering what they wanted to hear.

There were several teams working on this experience, and we were testing each surface (home, search, and library) in pieces rather than the full end-to-end experience a listener would go through.

One of the studies we pushed for was to conduct research on the holistic experience. Instead of continuing with usability testing, we leveraged a longitudinal approach, something that we didn’t do before for this study. We brought stakeholders from the different teams to see firsthand a new Spotify listeners’ journey as one collective experience. They were able to see how the features they worked on connected to other features, and the expectation they set for the overall experience — sometimes leading to delight and other times confusion.

The teams realized that an experience is made by the sum of its parts and course-corrected, bringing together their roadmaps and creating a new long-term strategic vision.

We wouldn’t have arrived here if we hadn’t taken a step back to ask ourselves:

• Are there opportunities we are not seeing by only focusing on the design or product feature?
• What emerging themes are we seeing that are worth further investigation?
• Do we want to continue to de-risk low risk features, or do we need to de-risk new opportunities and problems for the company?

If you’re finding it hard to move beyond feature improvements, ask yourself why?

• Is it a lack of context on larger business questions? If so, might be time to talk to leadership
• Is it lack of buy-in? If so, then do some one-off pieces and see if you can get others interested in the work.
• Is it a lack of resources? Hire a contractor, or democratize research.

Strategic Impact

How can we drive business decisions, while also ensuring research stays actionable?

On the other hand, when we are focused on blue-sky, ambiguous questions, our challenge now is to figure out how to make strategic insights and impact accessible. If we stay too high-level, we are unable to map the research back to existing roadmaps, and make it actionable for the teams building the product. With strategic impact, there needs to be a balance between long-term implications, and immediate actions the team can work on that ladders up to the overall vision.

We need to make sure that any ambiguous question is relevant to the team. At Spotify we did research on the sharing experience. Our team was working through the experience of a sharer and receiver. Given that there are several teams across Spotify interested in these learnings, we conducted a multi-market exploratory research diving into sharing behaviors and motivations. To make sure that the learnings were applicable to the immediate product team, we kept the approach grounded in the product experience.

We wanted to understand what was being shared across different audio services, while also looking at the underlying motivations and needs in the sharing experience. We were able to create a journey map of the sharing ecosystem, looking both at the user needs in the sharing and receiving experience, understanding why people share, key considerations when sharing, and existing barriers and opportunities. In highlighting the barriers, we could also make direct recommendations for the product team to address. By uncovering this ecosystem, we were able to explore new ways to approach existing problems differently. This gave us a clear picture of the pain points encountered in the experience — sharing was tied to more emotional needs like connection, and expression, whereas receiving was tied to more functional needs like access and content fit.

For strategic impact, it’s not just about delivering insights, and then letting the product teams figure it out. We still have to incorporate questions and processes that directly inform what the team works on in the short-term. It’s important that research stays accessible by the teams working day-to-day on a project. This means the way you tell the story of the insights should be inspirational, while also having immediate recommendations for the team.

Again to keep us focused on actionability, we asked ourselves:

• Is this research opportunity top of mind for your stakeholders?
• Are there other teams who will benefit from this research (e.g., marketing, business strategy, etc.)?
• Can the learnings from this research directly apply to what the team works on?
• How can we design research in a way that the team can grasp the research insights?

You can also read more here about creating strategic impact for your team through multidisciplinary research.

Perception Impact — How can we influence our stakeholders?

Although this type of impact is more often prioritized in smaller, and younger research organizations, it’s still important to tap into when looking at product/design impact, and strategic impact. Perception Impact hits on how we can influence the mindset of our stakeholders — either to get their buy-in for user research in general, or helping them have a new perspective on our users.

This impact starts with establishing and building trust with stakeholders. For this, we found success in partnering with related disciplines that are influencing key decisions. At Spotify, we put a lot of value on analytics and behavioral data, which is one of the reasons why our insights team includes both data scientists, and user researchers together. In having both disciplines working together we can appeal to what matters most to our stakeholders, and bring in both perspectives to help the team make informed decisions. Here are some questions we asked along the way to build this trusted relationship:

• How are our stakeholders arriving at their beliefs, assumptions, and hypotheses?
• How can we show stakeholders the risk of not addressing their beliefs, assumptions, and hypotheses?
• What value can we offer to showcase the benefit of conducting user research?

So what should I be doing to focus on more than one type of impact?

The key to amplifying impact is to comfortably move between these areas of impact (product and design, strategic, and perception) and knowing when you need each. A simple way to think about it is:

• Product and Design Impact helps with the next iteration
• Strategic Impact is for long-term goals
• Perception Impact is how we change the way we think about research, and people

Problem Space 2: Impact lives in one phase of research

It’s common to believe that changes from research are decided toward the end of a study — when we are in the process of report-writing, and presenting learnings back to stakeholders. However, if we incorporate impact-focused activities from the beginning, we are able to ensure that impact exists at every phase of research and after.

Here are some ways to keep users at the center of the research process, and create action-oriented activities that allow your study to have a greater impact.

Planning and scoping research

Firstly, you need to bring your team or stakeholders into planning and engaging with them way before we start conducting research.

We need to collaboratively work together with stakeholders to understand what they want to learn. If they are invested in the research goals, objectives, and questions, they are likely to be invested in the learnings and insights that come from research.

Useful Questions:

• What are your burning questions related to this problem/opportunity?
• What decision is this research informing?
• What do we need to de-risk? What is at risk if we don’t do this research?
• Why is this problem/opportunity important to learn?
• What assumptions are you making about people, process, and/or problems?
• What hypotheses are you trying to validate?

Activities / Tools:

• Collaborative research briefs: Document the research scope in a brief. Center the research goals, objectives, and questions around their needs. Use this as a tool to get everyone to agree on the overall idea of the research. Don’t be shy to incorporate your perspective into the brief — you are the research expert.
• Research kick-off: This is more of a formality but important to allocate the time for this in your study, especially for more expansive, foundational studies. This allows you to bring together your stakeholders so that they understand the approach.

Conducting research and fieldwork

Ask stakeholders to commit to observing research. Make it easy for them to join, and keep them accountable by assigning roles and responsibilities that make them a crucial part of research. If they are not going to play an active role, then push back. Research shouldn’t be an afterthought, and in order to have impact we have to keep stakeholders accountable and committed so that it stays prioritized.

Useful questions:

• What is sticking out to you from research?
• What feels in support or contradictory to what you believe?
• What participants are memorable to you? why?

Activities and tools:

• Note-taking grids: Get your stakeholders involved, give them the structure to help with their observation so they are actively listening. This can be done in a spreadsheet or an online whiteboard tool.
• Dedicated research Slack channels: Make research fun! Set up a forum that allows discussions to take place. Bring in context about the participant. Let people know when interviews are happening, and make sure they know the responsibilities they’ve committed to.
• Adopt-a-participant: Have each stakeholder immerse themselves with a specific participant. This is especially useful for longitudinal studies that follow participants over a certain period of time. Check out the blog post from Akshay Verma & Kathy Lin devoted to this process.

Synthesizing and analyzing findings

After fieldwork is over, you’re left with a lot of data. So, what do you do with all this information? The best place to start with synthesis and analysis is to return to your research brief. This will help you map a way through the data and understand what’s important.

When we think of analysis, researchers have a tendency to retreat, analyze on their own, and then share what they found in a research presentation. This hinders the potential impact of research because stakeholders may not grasp the most important insights or learning. They may only listen to what they already believe in. By opening up our analysis, there is a shared ownership of the insights between research and the stakeholders, they can internalize and own the learnings as well. We first need to let stakeholders into the process!

Useful questions:

• What were your research goals, objectives, and questions?
• What were the hypotheses or assumptions behind this research?
• What decisions will the research inform?

Activities and tools:

• Researcher sense-making: In the days following fieldwork, take the time to collate the data you’ve collected. This includes your notes, stakeholder notes (from the note-taking grid), and debrief notes.
• Immersion and analysis workshops: These workshops allow you to invite stakeholders into the process. Researchers take on the role of facilitators, we design a workshop that allows stakeholders to be close to the people, and the data. These workshops give us space to discuss any assumptions, what we learn from research, and challenge any closely held belief (especially when their assumptions and the realization from research differ). To run these workshops well, it’s important to package the data in a way that is digestible for stakeholders — like highlighting the voices of participants that really exemplify a key learning (from your initial sense-making), and point to key focus areas for discussion. This can also help teams prioritize what is the biggest problem worth solving.

Sharing and communicating insights

If we have incorporated efforts that amplify impact across the research journey, sharing and communicating insights will happen from the team and stakeholders, and not just fall solely on your shoulders. Stakeholders will be advocates for what they learn because they have been brought into the process from the start. They’ll be able to speak to user needs, and problems in the meetings they’re in that you may not be.

Instead of just sharing the insights, because we are already invested in these learnings, you can now think of this phase as “now what?”. What should we do now because of the learnings? These types of discussions ensure that your insights will be more than a static recommendation in a presentation.

Commitments from teams should be centered on implementation (if it’s product and design impact), teams will know what emerging opportunities they are willing to invest in (if it’s strategic impact), and mindsets and attitude around the value of research (perception impact) will be changed for the better.

To wrap up, we have to remind ourselves that we are not just researchers. We also have to be facilitators, educators, advocates, and program managers. We can turn these problems into opportunities, and think about how we can move fluidly between different types of impact, and incorporate impact at every phase. In doing so, we can maximize our impact and partner with product, design and tech to build great things.

Credits

Emily Chu
Insights Manager