ROC2023 Preview: Building a Triple Threat — Data Analysis, Organization-Wide Repository, and Education Library
By Hannah Campbell, Jeff Sokolov and Rachel Searing at iRhythm Technologies. Based on an upcoming ReOpsConference presentation.
Does your UX team want to showcase their work and contributions? Do you need to demonstrate how UX supports your organization’s goals? Do you want to educate cross-functional colleagues on UX and what we do?
As we were building our user research repository, we found countless resources on how to create a taxonomy and analyze data quickly, but limited advice when it came to ensuring the repository can be shared easily and meaningfully among colleagues and executives.
In our upcoming ReOps talk, “Building a Triple Threat — Data Analysis, Organization-Wide Repository, and Education Library,” we will detail how we not only provided the UX research team with a powerful qualitative data analysis tool, but also curated the information for simplicity and ease of use. We will share how we applied UX methodology to test our initial build, and how we are using this tool to educate stakeholders on the value UX brings to the product lifecycle.
This post focuses on the second pillar of our talk: how we applied user research methodology to create a usable and valuable research repository for our organization. In this post, we’ll take you through our experiences, from initial assumptions to the discoveries that led us to revamp our approach.
A Repository for All
Building a research repository from scratch is a formidable effort. Our journey began with several assumptions, on which we relied as we built the first draft of our research library.
Assumption 1: ‘Single Repository for All’
We began with the conviction that a “Single Repository for All” would be an invaluable resource for our research team, as well as our cross-functional colleagues and executive leadership. We believed that if executed correctly, the repository would become a one-stop hub for insights and an essential tool for the broader organization.
Assumption 2: Tailoring the homepage for new users
We assumed that tailoring the homepage to new visitors would provide the maximum value to our users. Our initial concept included sections like “What is user research and why is it important?”, “Highlighted Content of the Week” (to provide a starting point for visitors unsure of where to begin), and “Helpful Information for Newcomers.” We believed this approach would ease users into the repository and make it more user-friendly.
Assumption 3: Sorting research reports by year
When we were building the first iteration of our research repository, we chose to primarily sort research reports on the homepage by year, showcasing the most recent research first. We believed that this would be the most useful approach for our users and that it would ensure our cross-functional colleagues are up to date on the latest findings and insights.
Assumption 4: Empowering users through search
One of the things that drew us to the platform we are using for our repository was its powerful search feature. Beyond using the function within our research team, we believed the search feature would enable our colleagues to easily search across all our research artifacts. We believed that users would feel empowered by the search feature and the ability to search topics that interest them.
Learning Through User Research
Although we structured the initial iteration of our repository based on the assumptions listed above, as a team of researchers, we would be remiss if we did not involve users in the process. To validate these assumptions, we conducted user research in three parts: a survey, a round of usability testing, and an eye-tracking study.
Part 1: Survey
The first method we employed was a survey of potential users (a diverse group of cross-functional colleagues, with varying levels of interaction with UX). We aimed to assess the level of interest in our repository beyond the core product team.
We received over 100 responses to the survey and found that our colleagues, like us, also saw significant value in the concept of a “Single Repository for All.” Our first assumption was not just validated; it was amplified.
However, the survey results also revealed an important caveat. To make our repository genuinely valuable, the content within it must be fresh, updated, and presented in a clear, user-friendly manner. This became a crucial key to maintaining ongoing engagement.
Part 2: User Interviews
Following the survey, we conducted usability testing and interviews with 10 participants. Like the survey respondents, usability test participants overwhelmingly saw the value of a research repository and hoped it would have a broad organizational impact (this further corroborated our first assumption). They indicated that, if executed properly, a repository could reduce reliance on institutional knowledge, demonstrate the broad impact of UX research, and increase cross-departmental collaboration and information sharing.
However, usability challenges within the repository became evident. Our third assumption led us to organize research projects in the repository primarily by year; however, through usability testing, we learned that users expected research to be sorted by product and project topic, rather than the date it was executed. Although the repository allowed users to find research-by-topic, it wasn’t supporting that use case efficiently or effectively.
Additionally, we had assumed that offering detailed guidance on how to use our repository would make the platform more user-friendly (assumption 2). The reality became clear in the usability testing: our users were ready to explore, and our detailed instructions seemed like barriers to entry.
We also learned that features innate to the platform were causing confusion. The platform we built our repository in automatically surfaces the first ~500 characters of a research report before a user clicks in. This led to confusion; users expected surfaced text to be a proper abstract, but instead, it simply pulled the first few sentences of the report. Moreover, we had assumed the search feature innate to the platform would empower users (assumption 4); in reality, it was deemed complex and too confusing to utilize meaningfully.
Part 3: Eye-tracking
To round out our initial research efforts, we executed an eye-tracking study. Like the participants in the usability test, participants in the eye-tracking study also indicated that finding research by product or project topic is their primary use-case (challenging our assumption 3). Although the repository was primarily sorted by year, we believed it also enabled users to browse by research topic. Through eye-tracking, we were able to discover that the research-by-topic areas were “blind spots” to some participants — their eyes fixated on those areas, but they had no memory of having seen them and were unable to complete tasks related to them. This was a clear indicator to us that we needed to reorganize the home-page of our repository, to enable users to efficiently and effectively browse research-by-topic.
Tips & Takeaways
The insights we uncovered through research challenged some of our initial assumptions, but it also provided us with a roadmap for improvement. Instead of organizing research reports by year (assumption 3), we have reorganized them by product topic. Instead of overwhelming users with help content (assumption 2), we have provided what we believe to be the bare minimum needed to use the repository efficiently. Instead of relying on the platform’s complex innate features (assumption 4), we’ve built workarounds to ensure the experience of navigating the repository is easy and intuitive.
If you and your team are also feeling stuck or overwhelmed when ensuring your repository will be invaluable for your entire organization, we hope the following tips and takeaways resonate with you.
Takeaway 1: You have to start somewhere
When faced with a formidable, daunting task like building a research repository from scratch, it can be overwhelming to know where to begin. For our team, the quickest and most effective way to move forward was to begin with assumptions — and although this might not be the best way to start, it’s better than not starting at all.
Takeaway 2: Get your users involved
Say it with me: “users” doesn’t always mean “customers.” For a field that is so vocal about putting the user first, we often forget that users can also be ourselves and our colleagues. If you were building a content library for your customers, you wouldn’t hesitate about getting users involved in the process– don’t forget to do the same for your repository! This becomes especially important if you begin assembling your repository based on assumptions, as those assumptions may be proven wrong (as some of ours were).
Takeaway 3: Approach constraints with creativity
The platform you choose to host your repository will inherently have constraints. Rather than seeing constraints as obstacles to overcome, we encourage you to think creatively about how you can use them to your advantage. As discussed earlier, the platform we built our repository in automatically surfaces the first ~500 characters of a research report. In our usability testing, we uncovered that users expected surfaced text to be a proper abstract, rather than the first few sentences from the beginning of the report. From this insight, we have revisited how we structure research reports, such that the first ~500 characters are an abstract. Using the constraints of the platform to our advantage has ensured that we’re able to meet users’ expectations and provide an overall better experience.
Takeaway 4: Meet your users where they are
Any seasoned UX professional will tell you that it can be difficult to change user behaviors. When it comes to building an internal repository, we believe the most efficient path to adoption and engagement is to meet your users where they are. If your users will not take the time to read your in-depth help content, we encourage revising it to the bare minimum. If your users are overwhelmed by content and are looking for summaries of research rather than in-depth reports, we recommend finding a way to provide that summary. If your users want to search but are overwhelmed by the native feature, we recommend building a workaround. You only get one chance to make a first impression, and meeting your users where they are is our recommendation for ensuring adoption and engagement of a research repository.
Thanks for reading! Learn more at the ResearchOps Conference 2023
Rachel, Jeff, and Hannah will be expanding on these ideas (and more!) in their presentation at https://reopsconference.com/ on Thursday, 16 November 2023.
