Activating insights to overcome common barriers to product impact

Image: Adrian Howard

This article is based on a talk of the same name at the inaugural ReOps Conference in June 2022.
Watch video of talk on Learners website.

In this article:

• The rise of research knowledge management
• Setting realistic expectations about research repositories as internal tools
• Research findability is a foundation for what’s possible
• Modeling the path from insights to product impact
• Which research should we try to push back into product conversations?
• Prerequisite: Creating an inventory of problem-to-solve insights
• Prerequisite: Surfacing the ‘cream of the crop’ in your insight inventory
• Prerequisite: Assigning potential product team ‘owners’ to insights in your inventory
• Stage 1. Sufficient evidence
• Stage 2. Usefully articulated insight
• Stage 3. Awareness of an insight as a possible planning target
• Stage 4. Envisioned solution ideas
• Stage 5. Prioritized plans
• Stages 6. Quality execution + 7. Understood results
• Takeaways from exploring these new operations

—

The rise of research knowledge management

ResearchOps is a big space of activities, and the knowledge management pillar is a fast evolving area [1], with plenty of room to grow.

Collections of unapplied insights build up quickly in tech organizations. As streams of research are conducted, key insights for better serving people are too easily left on the cutting room floor, piled away in locations that product people do not frequent. If your organization has been around a minute, then chances are good that your past research could have a lot more to say about your future product direction.

These days, the industry is focused on research repositories as a means to fix the cutting room floor problem, consolidating research outputs and making them findable. But repositories, on their own, do not make past insights come to life. A tool, by itself, is not going to change product planning processes.

Repositories can, however, enable new pathways to meaningfully push past learning. Teams can envision new research operations that build outward from their repositories to push insights back into product conversations.

Setting realistic expectations about research repositories as internal tools

Let’s do a little thought experiment: imagine the perfect research repository. Wow, look at that beautiful metadata! And the research community is actually putting things in there and collaborating like they said they wanted to. Contributors are working through their fears about opening the doors to their research — getting out of the middle of access to their insights and building new connections with product people who are interested in their learning [2]. You’ve integrated a lot of content into your repository tool, and now it’s findable. That’s no small task. But is that the end of the line? What about integrating that great research content into actual product planning?

Some product people — be they designers, product managers, engineering leaders, marketing leads or otherwise — are logging into your tool and working insights into their plans. And that’s big news. Another level of impact unlocked.

But to achieve the level of impact that your research community wants to see, how many people would need to visit your repository? Who would have to opt to make your resource part of their workflow? How often? Is it realistic to expect your internal tool, among a range of other internal tools, to have this kind of pull?

What about that massive pile of unaddressed insights that aren’t getting the attention they deserve? Do we leave it to product people to ‘leverage’ our tool as they see fit? Or do we use our repository as an enabler of more intensive follow through on research, pushing a clear, evidence-based point of view on what’s important?

What happens when a senior leader sees the cost of your repository program and wants to know, specifically, what impact it’s having on their products?

Research findability is a foundation for what’s possible

As research teams aggregate and synthesize studies within repositories, they eventually confront the reality that just because insights are more available, they are not guaranteed to have impact on product. If we’re not careful, we’re just putting a lot of effort into consolidating and organizing the cutting room floor.

It’s time to set a higher definition of success and to think bigger about insight debt. Our goals for research repositories can be more than making insights findable; we can use these internal tools to drive research visibility and accountability in planning. Not just informing roadmaps, but pushing for changes and new product directions.

If you already have some sort of repository tooling in place, this article will suggest a bigger frame for what you could be doing with your repository programs. And for all those folks who don’t have great repositories in place yet, but want to, I’ll be suggesting operations that you could get started on ahead of any new repository efforts — which can help you define key requirements for your real tooling needs.

Modeling the path from insights to product impact

Looking back at past efforts to reactivate research, it’s clear that there are some common barriers on the path from a pool of existing insights toward corresponding launches of features, experiments, and services. With the goal of finding new ways to overcome those barriers, here is a model of some high-level stages:

First there’s a category called ‘Integrating research content,’ which includes: 1) being perceived as having sufficient evidence, 2) having a usefully articulated insight as a problem to solve that product people can address, and 3) awareness by product people of the insight as a possible planning target, which spans into the next category: ‘Integrating into product planning.’ So once you have all that research content integrated together, what does it need to fuel in order to make a difference? This category includes, 4) being able to envision solution ideas from an insight, 5) getting ideas transformed into prioritized plans, 6) quality execution that will deliver on an idea’s promise, and 7) understood results, which can then feed back into sufficient evidence as the process starts again.

We’ll dive into operational ideas for each of these stages. But first, let’s check in on some ‘prerequisites’ we need to tackle before trying out new operations for advancing past insights.

Which research should we try to push back into product conversations?

But hold on, you might be asking: Aren’t there many types of insights and impacts? Can they all be pushed in this way? It’s a good question.

Let me start my answer by saying that there’s no substitute for great research practice at the individual study level. Bringing teams into synchronous study activities and activating insights while researchers are still deeply involved. Participation that leads to minds shifted, features conceived, designs enhanced, and issues prevented from launching. There are many types of participatory impact that can occur during the timeframe of data collection and analysis that are difficult to recreate after the fact.

But if you take a highlighter to past studies and mark up which insights actually impacted products, that cruel reality of the cutting room floor awaits. There’s typically a bumper crop of crucial insights that have not yet fulfilled their potential as launched offerings.

When studies are done and dusted, and research operations is working to pick up existing learning that was not acted upon, I would argue that we will get more value out of focusing on certain types of unaddressed insights. Rather than emphasizing existing insights that can build general understanding of customers and their worlds, we can narrow in on the subset of insights — from generative, evaluative, and other studies — that can more directly drive launched improvements.

When you’re looking at piles of unaddressed research and deciding what to reactivate, insights that can be framed as evidence-based ‘problems to solve’ have the clearest pathway forward. In an organization that’s iteratively innovating, there are 1) problems that could lead to launching new things toward entirely new needs, and there are 2) problems that could drive iteration to better serve a need that your organization is already striving to serve.

If all that sounds a bit reductive, I’m not suggesting that this kind of post-study activation of ‘problems to solve’ insights is the only valuable thing. Far from it. And every research program to reactivate existing research should find ways to increase first-hand participation in upcoming research projects in order to get product people closer to real people. However, for the purpose of this article, let’s narrow our focus on how existing ‘problem type’ insights can lead to traceable impact in product plans.

Prerequisite: Creating an inventory of problem-to-solve insights

When I talk to folks who are thinking of investing effort in a research repository, they often want to know whether it’s okay to stay with a ‘register of reports’ or whether an ‘insight hub’ is really needed [3]. In other words, whether they should put the effort into ‘atomic research,’ where individual elements from a study are captured in a tool as distinct, referenceable objects [4].

When reactivating existing research in planning, you need to be able to point product people to individual insights as potential problems to solve. But to get started experimenting with new operations, you do not need to invest in a fancy insight-level repository tool — getting to an inventory of linkable, individual insights can be accomplished in a variety of ways.

An inventory of insights can be a list from a meta analysis on a crucial topic, captured in a spreadsheet; or a deep dive with each researcher about their most important insights from that have not been acted upon, captured in a wiki; or ‘top findings’ from the last three years worth of reports, captured in an Airtable. Or, on the more effort-intensive end, your inventory might be every problem-to-solve insight from past studies, pulled from existing reports and documented in one of those insight-level repository tools that’s now available in the market.

Regardless of where they live, insights in your inventory will need to have unique identifiers and be addressable. Something specific that can be linked to, even if those links just go to a spreadsheet, which then deep links into evidence in corresponding source reports.

Prerequisite: Surfacing the ‘cream of the crop’ in your insight inventory

Assuming that your research organization has been up and running for a while, your inventory of existing problem-to-solve insights can become quite long before you know it. Since product development resources are limited, you simply can’t push every unaddressed ‘problem type’ insight through toward launch. You’ll need to develop a rubric and rank your insight inventory.

The need to rank insights was driven home to me when I returned to a ResearchOps job (after taking some time off to be a full-time parent) with the intent to reboot an ongoing repository initiative.

Product managers were pushing back — not about the value of the insight repository — but about the need to know which insights were most important. There was just too much to sort through. We workshopped and applied a new rubric to thousands of insights. It was nerve-wracking, but it turned out that our ranking was rarely questioned over a timescale of years.

There are many possible approaches to ranking a set of insights, and workshopping your rubric ideas can help build buy-in for next steps. Even basic ranking schemes can quickly surface the ‘cream of the crop’ that you can focus on reactivating, while at the same time pushing another set of insights to the bottom of your inventory. Customer-centered factors — such as estimated frequency and potential impact on customer trust — are a clear fit. And some folks also factor in an insight’s potential contributions to organizational goals or the possibility and scale of business wins. Not from exacting business intelligence, but from general rules of thumb. The type of heuristics that researchers often apply when deciding what to lead with in their individual studies.

And let’s be clear — even the ‘cream of the crop’ of insights in your ranked inventory is a mountain to climb. Your ranking is about focusing research operations’ own activation efforts, with the hope that these efforts will inspire design and change the plans of various product teams. But this is a ‘long game,’ and not everything is going to happen right away — just talk to product owners about all the things in their backlogs they wish they could get to!

Prerequisite: Assigning potential product team ‘owners’ to insights in your inventory

Once you have a ranked subset of insights in your inventory that you’re especially focused on, you’ll need to tag one or more potential product ‘owners’ to each insight. Who might act on an individual insight? Who should the insight connect with? This is a chance to route insights to more teams than had originally seen them, expanding the ‘impact radius’ of specific problems to solve [5].

With this tagging in mind, maintaining a map of teams and product owners is a necessary reality, even though it is difficult ongoing work.

Tagging product teams — or some other level of potential insight ‘owner’ within an organization’s chart — is the most important metadata for getting things done with insights.

Okay, so wrapping up our prerequisites before exploring new operations to advance past insights: once you have 1) an inventory of ‘problem to solve’ type insights that can be linked to, 2) that inventory is ranked to surface a point of view on which insights are most important, and 3) these insights have been individually tagged to potential ‘owners,’ you’re ready to think about pushing forward. Now, to better understand what pushing insights toward planning might look like, let’s dig into each of the stages of our model.

Stage 1. Sufficient evidence

Being seen as having enough data to justify action.

The key words here are being ‘seen’ as having ‘enough’ data. In spite of all the data theater, many product prioritization decisions are based on leaders’ hunches and interests. Think back to early product launches in your organization, when most everything was based on bets from a small amount of compelling evidence about a real customer problem. How quickly that can change as organizations start to tilt toward ongoing management over founding leadership.

Ideas for research operations at this stage include:

• Regrowing norms about big bets from inspiring evidence
  Everyone encounters those cases where a small sample qualitative insight — an obviously valuable problem to solve — goes against the grain of product owners’ thinking, and they’re just not buying in. These arguments at the individual insight level can be challenging, but you can work to change product planning culture over time and re-establish earlier norms. Try collecting and evangelizing internal stories where respected leaders made big bets from a small amount of inspiring evidence — and launched big wins.
• Investing in targeted synthesis from across teams and sources
  For crucial insights that are seen as needing more evidence, additional data can often be found in other internal teams and sources. Every time I start working on a launched product, I always find tons of unanalyzed data, as well as specifically useful research that never really saw the light of day. Strengthening the persuasiveness of existing insights sometimes means putting in the interpretive labor to synthesize scattered evidence together.
• Driving research roadmaps to collect more data in key areas
  Inevitably, there will be some insights where you just can’t find any other evidence around your organization. In these situations, it may be worth steering upcoming research efforts to collect more data. Sharing a list of insights that need additional support can prime researchers to keep an eye out and report back, while also providing research leaders with clear targets for their teams’ study roadmaps. Again, keep in mind that pushing past insights forward is a ‘long game.’ As new evidence appears in incoming research reports, you can add it to the insights in your inventory. And each time there’s a new story to tell for one of your top insights, you can treat it as an opportunity to reconnect with ‘owning’ product teams.

Stage 2. Usefully articulated insight

Getting to the ‘right’ problem to solve.

At this point, I’ve probably consumed thousands of research reports, from a variety of different disciplines and organization types. Researchers have many different styles for writing headline findings that are intended to drive product action. And to be frank: some approaches to articulating insights are much more effective than others. What’s the best way to describe an evidence-based problem to solve in order to make it clear and compelling for intended audiences in your organization?

Ideas for research operations at this stage include:

• Insight writing and content standards
  Research operations can convene insight generators to talk about what’s worked and what hasn’t worked when communicating insights. A working group can use these learnings to develop standards for clear problems to solve that inspire action. These standards can take the form of some recommended guidelines or be as strict as a form fill-in UI in an insight hub repository. Such rules may sound trivial, but they are not. When you have a list of existing insights, standards are about making them parallel — even though they are from different researchers, projects, and topics — so that product people can grasp them more readily.
• Hierarchical insight themes, from forest to trees
  Sometimes getting to a useful articulation means zooming out to show the areas of the ‘forest’ in addition to all the individual insight ‘trees.’ You can work with your researchers to build meaningful hierarchy into your ranked insight inventory, not so different from the structure of a large-scale, generative research report. Make sure your higher-level insight themes are also framed problems to solve, rather than general buckets such as stages of a customer journey.
• Matching insight granularity to targeted level of planning process
  Once you have a hierarchy of different levels of problems to solve, you can better tailor your ranked inventory to different types of planning activities. For example, at higher levels of the planning process — as in a yearly company-wide plan reviewed by top leaders — zooming out to bigger picture problems to solve may be the most useful. On the other side of the spectrum, when working with individual product teams around their backlogs, you may find that granular insights are a better fit — while referencing larger themes that put individual insights in context.

Stage 3. Awareness of an insight as a possible planning target

Building mindshare for crucial insights.

A good articulation of an evidence-based problem to solve allows it to be picked up, but the right product owners still need to know about it in order to consider it as a planning target. And knowing about it doesn’t mean having heard it once. It’s useful to think of this communication work as internal marketing of essential insights.

Ideas for research operations at this stage include:

• Recurring ‘echo’ report readouts
  Even when insights find the right audience, there are so many cases when it’s the wrong time for teams to actually consider them. Everyone nods and there’s general agreement, but since the team is currently working on something entirely different, next steps get lost in the ‘good intentions’ shuffle. For reports that contain top insights from your ranked inventory, research operations can schedule ‘echo’ discussions of a study with relevant product people. These ‘echos’ can include any new evidence for the top insights in question, while focusing conversations on next steps and resulting plans [6].
• Pushing new types of reporting
  In addition to ‘echoing’ whole reports that contain top insights, you can use your insight inventory as the basis of new communications. Try crafting new types of reporting built from filtered sets of cross-study insights, packaged up for specific teams who might act on them. Conduct marketing experiments within what seems native to your planning culture. That could mean regular focused meetings around walls of exported insights; a weekly insight post in Slack to keep eyes on your inventory; or a slot in leadership meetings where progress, areas of improvement, and accountability are discussed.
• Timing communications to specific planning milestones
  And the real key to this type of insight marketing is timing. Matching report ‘echos’ and ‘new types of reporting’ to the actual timeframes of planning, attached to specific product milestones. Great timing is about syncing to the mindset of product leaders — when leaders are currently executing on existing plans, they are often not really mentally prepared to think about new ideas at a meaningful level of detail. But when they are stepping back to dream as part of their own design or planning cycle, they will process evidence-based problems to solve totally differently [7].

Stage 4. Envisioned solution ideas

Bridging from insight to understood effort that could be prioritized.

It’s one thing for product people to be aware of an insight, but it’s quite another for leaders to have an idea about how, specifically, they might address it with launched experiments, features, or services. Some evidence-based problems to solve have clear next steps; others present an ambiguous fog that clouds what could come next.

Ideas for research operations at this stage include:

• Securing design resources to visualize ‘in the air’ solutions
  Some insights in your inventory will have solution ideas that have already been discussed repeatedly — but haven’t yet landed in product plans. When asked about the problem to solve, everyone imagines a similar next step, but these ‘in the air’ solutions somehow haven’t made the cut. Research operations can unblock progress by working with design leadership to secure design staffing that can visualize ‘in the air’ ideas as concrete, read-to-estimate proposals.
• Facilitating and fueling ideation for non-obvious problems
  It goes without saying that not every research-based problem to solve has an obvious solution. Programs to reactivate existing insights can draw from the same ideation toolbox that research teams use to activate learning at the end of their studies. And there’s no substitute for securing the efforts of energetic designers who can work ahead and generate concepts for crucial insights, removing the fog so that product owners can imagine new user experiences.
• Cross-linking solution proposals to insights
  When ideas to address insights come together, you can link them into your insight inventory — and vice versa — so that solution and research-based rationale are paired. And by keeping track of insights that need design or have designs in place, you can get a sense of how your ranked insight inventory is progressing. That progress is something to highlight as you work to keep leaders aware of your insight activation program and continue to grow buy-in.

Stage 5. Prioritized plans

Marshaling limited implementation resources (owned by others).

Once it’s clear what could be prioritized to address an insight’s problem to solve, product owners still need to capture related proposals in their own planning workspaces and actually commit limited engineering time to bring ideas to life. Similar to the earlier stages, there’s no magic recipe here — especially given that product disciplines work so differently in different organizations.

Ideas for research operations at this stage include:

• Driving specific backlog items that address insights
  In some product cultures, getting into the backlogs of targeted teams is a necessary step for prioritization to occur, documenting work items in a format that makes sense to implementers. I’ve spent a lot of time ‘priming the pump’ with teams by mapping insights directly into their existing backlogs and roadmaps — turning their work products into research repositories of sorts. And many times, related planning pieces are already captured somewhere, especially if they were defined just a bit differently in conversation with product owners. Outside of taking on this work team-by-team, big cross-team efforts to generate new backlog items en masse will require high-level leadership buy-in in order to drive real participation from busy product leaders.
• Tying into primary goal setting levers and creating ‘blanks’ that product teams need to fill in
  It goes without saying that just because a plan is documented doesn’t guarantee it will receive a single engineering hour. What really matters is the front edge of a product team’s roadmap, where competing priorities meet. Every organization has some way of setting goals that product leaders are held accountable to. Learning about and tying into goal-setting mechanisms can help transform acting on insights from a series of one-off conversations to something with a stronger systemic footing. Goals can be about tackling a number of things from your insight inventory — but simple counts may result in teams picking off smaller items. Even better, insight-driven goals can read like any other goal in a product team’s list, but with the acknowledgement that they address specific, evidence-based problems from your ranked inventory. A bold move into product operations can be as straightforward as getting leadership buy-in to create a planning ‘blank’ that product owners need to fill in. A space where owners list which insights from your inventory they are targeting with their goals, key results, or whichever format your organization uses.
• Reporting on action and inaction to grow norms of insight use
  As you’re continuing to increase mind share about your inventory, you can celebrate planning against insights in a very visible way, growing norms for research use. Crow about actions by product people that you want to see replicated in other teams. And on the stick side of things, you can report on inaction in an objective way that is visible to leadership, giving them the option to respond as they say fit. These spotlights can occur in standing business review meetings or other channels — not so different from the ‘notes from the field’ that a researcher might send to keep folks plugged in on progress. Overall, these reporting efforts are another form of internal marketing, helping your organization keep a finger on the pulse of your insight activation effort.

Stages 6. Quality execution + 7. Understood results

Staying connected to implementation and measurement.

Even after work targeting insights is prioritized and in flight, there can still be reasons to stay connected to the process.

Ideas for research operations at these stages include:

• Bringing new folks into the problem space
  As work moves from plan to implementation, more people join the effort to address an insight. For crucial insights in your inventory, consider giving ‘echo’ readouts to engineers and other staff who are new to the problem being solved. Immersion in the existing research can help make sure everyone is clear on ‘the why’ and ‘what good would look like.’
• Checking in on quality assurance and experiment planning
  For especially important and high-risk efforts, staying connected may involve rallying the right staffing to ensure that insight-driven experiments are launched in their best form.
• Tabulating impact rooted in research
  As features and offerings launch, there’s value in putting on our bean counter hats to do some innovation accounting. Two direct measures are 1) the number of launches that referenced your ranked insight inventory, and 2) the metrics improvements from those launches that product teams have claimed as their own successes. As you’re adding up these wins over time, you can close out related problems to solve in your inventory, where appropriate. And given that this is a ‘long game,’ shouting about research-based launches can continue to grow norms for using research as a foundation in the product planning process.

Takeaways from exploring these new operations

So, we’ve explored some operational ideas at each stage in our model, bridging from existing insights to product impact. There’s clearly a lot more that could be said about each stage, but let’s step up a level as we conclude. What are some bigger-picture takeaways from this sort of shepherding of existing insights?

First, from the range of activities described above, it’s clear that the conventional end of a research process is actually an early milestone in a much longer view of shaping product plans. There are many more insight touch points to experiment with and land effectively with product teams.

We’ve also talked about how inventories of insights — whether in a simple list or in a fancy repository tool — can be brought to life through new operations. And as we push insights down the path toward product impact, we’re finding new ways to transform research from an optional input to an active stakeholder.

In this way, we can continue to expand how an organization thinks about its researchers. Operations can grow research’s seat at the product planning table, and clarify what researchers are asking for as we take that seat.

And while sufficient staffing from research and research operations could shoulder this insight activation work — keeping it solely within research teams misses the point. When activating insights, there are many touch points with the ‘next’ disciplines in the planning process. Staffing commitments from design and product teams can increase their stake in making ongoing insight activation work. You know how we’re always talking about increasing participation in individual research studies as a means to getting more done? The same is true of a collection of existing insights — more ownership from other disciplines will lead to more and better action being taken.

At a more basic level, shepherding insights toward impact presents a different frame for research repositories. Yes, research repositories can be self-service hubs, where product people find insights to inform their work. That fits a common conception of what this class of tools has become. But, as we’ve seen, self-service findability is just table stakes — repositories can become hubs to repeatedly push specific insights to product owners via new operations.

And with all of that push, when it’s successful, ranked insights from repositories become part of product teams’ own tools and work spaces — they become deep-linked rationale that’s integrated into proposals for what’s next. Your repository, regardless of how it’s implemented, extends beyond its tool-based container to live in a network of planning work products and practices. And a linked insight, connected into others’ outputs, is definitely no longer on the cutting room floor.

—

If you’ve read this far, please don’t be a stranger. I’m very curious to hear about your challenges and approaches for reactivating existing insights in your organization. Thank you!

Connect on LinkedIn
Sign up for email updates (monthly, at most)

—

[1] “The Eight Pillars of User Research” Emma Boulton
https://medium.com/researchops-community/the-eight-pillars-of-user-research-1bcd2820d75a

[2] “Opening the gates: Addressing concerns about research repositories broadening access to insights”
https://medium.com/researchops-community/opening-the-gates-addressing-concerns-about-research-repositories-broadening-access-to-insights-7baa6f5384b0

[3] “Research Repositories: A ResearchOps Community Program of Work” Brigette Metzler, Bri Norton, Dana Chrisfield, Mark McElhaw
https://medium.com/researchops-community/research-repositories-a-researchops-community-program-of-work-811b2ba3638f

[4] “The atomic unit of a research insight” Tomer Sharon
https://tsharon.medium.com/the-atomic-unit-of-a-research-insight-7bf13ec8fabe

[5] “Growing research ‘impact radius’ by connecting learning to more internal product audiences — to get more done with insights”
https://medium.com/integrating-research/growing-research-impact-radius-by-connecting-learning-to-more-internal-product-audiences-to-1eaf0fcc4ad

[6] “Adding ‘echo read outs’ to re-engage product teams with research projects — and get more done with insights”
https://medium.com/integrating-research/adding-echo-read-outs-to-re-engage-product-teams-with-research-projects-and-get-more-done-with-f60e7d0e0d2a

[7] “Extending insight ‘shelf life’ to get more value from research in product planning”
https://medium.com/integrating-research/extending-insight-shelf-life-to-get-more-value-from-research-in-product-planning-c2fb7af5d42a