Pushing for citations of research repositories in product practice — to get more done with insights

Citing evidence-based problems to solve and other inspiring insights from research repositories In product teams’ own deliverables. Illustration shows insight consumers’ work products citing specific insights in a repository.

Where are insights being used? How can we grow adoption of existing research if we don’t track its application? Product teams may be consuming insights, but the ‘through line’ to resulting action is easily lost.

Researchers can push new standards for how and where their research repositories should be referenced. Product people can then apply these standards to cite the insights they’re working against, communicating evidence-based rationale and unlocking other downstream value.

Even when research repositories are well-established and building up amazing content — if no one has a clear understanding of where those insights are flowing, they may be viewed as dead ends.

Speech bubbles: Q: “It’s so great that you’re working on features to solve that problem! Have you cited the insight in your outputs?” A: “No, but I will. How do I do that?” Q: “There’s a few different ways, depending on how much evidence you want to include. Here’s a link to our citation patterns.” A: “I’ll check it out and add it in. I like the idea of making the case with the insights that we’re working to solve.”

Citation in product development isn’t an academic problem; it’s about research making more of a difference for the people we’re striving to serve. It’s about visible connective tissue between insights and outcome — maintaining a line from spark to feature.

Improving your insight operations

  • Building the muscle of researchers citing themselves
    Request that your research contributor community cite existing insights from their own repositories — but don’t give too many specifics as to how. Let the experimentation run free at first, then harvest the different formats that contributors generate.
  • Defining standard patterns for citing repository content
    Bring together research leaders and repository lead users to review ideas for citation formats. Collect a range of product teams’ deliverables and discuss which product development locations would be ideal for research citations to have impact. Create citation patterns that could fit within these locations, ranging from minimal, named links to more descriptive formats — such as an ‘embedded insight’ that includes anonymized customer quotes, related links, and other content. Explore language for cases where insights are being ‘solved’ versus cases where insights are only ‘informing.’ Develop strategies to ensure that citation links will not turn into dead ends over time. Consider how your new patterns can reflect a larger brand for existing research.
  • Getting leadership buy in for new citation approach within targeted planning and design deliverables
    Shop around your new standards and assemble some early examples. Choose specific processes and templates to pilot your new citation patterns in planning and design work. These targets could be backlog item input forms, appendices in planning documents, introductions for design presentations, A/B experiment reports, or any other pivotal location. Mock up what ‘good’ would look like, and bring your proposals to various leaders. Start your presentations with a basic demo of your research repositories, sharing notable wins to date that leveraged existing research. Ask leaders to support the changes needed to enable citations — including updates to tooling and workflows.
  • Marketing the expectation of research citations and where they fit
    As part of communication campaigns from your repository initiatives (B2), share launched changes to templates and processes. Frame new expectations to cite research in terms of the value on offer, with documented insights inspiring and justifying product proposals. Ask your leadership and sponsors to broadly communicate about these new expectations. Offer pathways for product people to get their hands on existing insights for their areas, easily add insights to projects, participate in research processes, or even submit questions, if applicable. Ask researchers for help pushing out new citation standards, requesting payment ‘in kind’ for insights delivered in the form of citations within product deliverables.
  • Inspecting, iterating, amplifying, and expanding use of repository citations
    Search internal tools for citations, sharing new references with your research contributor community. Explore automation for these monitoring and reporting efforts. Evaluate where any citation blockers are occurring and iterate your standards and approaches (e.g. could other deliverables or processes be more impactful targets?) Provide social proof of citation best practices by including related wins in your research marketing campaigns (B2) and other high-visibility communications. Update your leadership on progress and challenges, reiterating the need for them to regularly voice their expectation to see research references in the deliverables that they review.
  • Your idea here…

On the path from insight to product impact

A diagram of seven stages on the path from insight to product impact: Category called “Integrating research content” 1) Sufficient evidence (grayed out), 2) Usefully articulated insight (grayed out), 3) Awareness of possible planning target (highlighted), Category called “Integrating into product planning” 4) Envisioned solution ideas (partially highlighted), 5) Prioritized plan (partially highlighted), 6) Quality execution (grayed out), 7) Understood results (grayed out).

Let’s connect

Related posts

Selected references

  • “When talking about measuring the impact, we have pretty good analytics so we can see whether people are reading stuff or linking to pages… The last part I’d add is — it helps if there is a leader in the organization who’s creating some pressure for people to either be consuming or linking to the research. If there’s a leader in the organization who says something around not wanting to see any proposals for new features without links to good research. Something as simple as that can create a habit where non-researchers are incentivized to show the research behind, or justify, why they’re making a decision. The impact this can have is, all of a sudden your research knowledge management tool is an asset to them and solving a problem that they have.” Matt Duignan, Lisa Nguyen
    https://dovetailapp.com/blog/microsoft-hits-question-answer-matt-duignan/
  • “The good news is that people are very willing to help, I find, even if they don’t know you, even if they outrank you on the org chart. If you succinctly explain to them what you’re trying to do, I find that people almost always reply and help me. Demonstrate that you have an agenda, clearly communicate the outcome that you want, and you’ll start to build trust. I think a lot of this is simply about building trusting relationships both externally and internally.” Hana Nagel, Sofia Quintero
    https://medium.com/enjoyhq/researchops-spotlight-hana-nagel-lead-ux-researcher-at-sap-534a8e3d89c1
  • “When measuring outcomes, moving beyond measuring whether research knowledge is used in decision making to measuring how research knowledge is used becomes important (Pelz 1978; Weiss 1979). Research knowledge may be used in instrumental, conceptual, or symbolic ways. Instrumental use is defined as acting on research in specific and direct ways, such as to solve a particular problem at hand (e.g., developing the first iteration of Medicare’s “Resource-Based Relative Value Scale” physician fee schedule). Conceptual use involves a more general and indirect form of enlightenment (e.g., resisting a move toward more for-profit hospitals because of a general sense that not-for-profit hospitals offer a survival advantage for patients compared with for-profit hospitals, but without knowing about the particular studies or their strengths and limitations). Conversely, symbolic use pertains to a use of research knowledge, but not to inform decision making; here research knowledge is used to justify a position or action that has already been taken for other reasons (sometimes called a “political use of research”) or the fact that research is being done is used to justify inaction on other fronts (called a “tactical use of research”).
    John N Lavis, Dave Robertson, Jennifer M Woodside, Christopher B McLeod, and Julia Abelson
    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2690219/

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jake Burghardt

Jake Burghardt

Focused on integrating streams of customer-centered research and data analyses into product operations, plans, and designs. www.linkedin.com/in/jakeburghardt