Define the North Star when analyzing and interpreting user data
After spending a few weeks conducting study sessions where we talked to users of the target population, user data has been accumulating. However, large amounts of user data can cause mixed feelings to researchers. On the one hand, researchers are excited about solid insights that will be uncovered, but on the other hand, researchers might feel overwhelmed by the nebulous look of user data. I am one of those researchers who experience the conflicting emotions when being about to analyze and interpret data. Nevertheless, I’ve tried things that effectively help turn the nebulous look into a clear vein. I refer to those helpful solutions as the “North Star” that guides me through the data analysis and interpretation process.
In this article, I’d like to share my approach to defining the North Star when analyzing and interpreting user data.
Create a place to scan all raw data
Most of the user data is collected via either recordings or questionnaires. To allow scanning the whole data of a project, a data dump needs to be created at the beginning of data analysis and interpretation. Specifically, users’ answers (either qualitative or quantitative, both), researchers’ observations, and other relevant stuff are put together in a place (spreadsheet). The complete view of raw data makes it easy to get a sense of how data were gathered for different research questions or if any questions didn’t get responses. Also, it is faster to start analyzing certain parts of data from the all-data spreadsheet.
Start with users’ profile data
Based on either the research plan or impressive moments that happened in collecting data, researchers sometimes directly start analyzing the part that draws most of their attention. Nonetheless, the effectiveness of data analysis can be affected by the understanding of users’ profile data. For example, the certain areas or clients with which users work lead to totally different reactions to some feature of a product. Therefore, to account for the influence of contextual factors (roles, job responsibilities, the level of experience, etc.), I start with analyzing users’ profile data before delving into certain parts. Another reason behind the idea is a good understanding of users makes data interpretation compelling.
Use notes to parse critical information
As mentioned earlier, researchers might feel lost when facing a large amount of user data. In the meanwhile, given that delivering research findings is time-sensitive, it might be unrealistic to uncover insights by going through all data. The notes that were either taken by myself or stakeholders who observed study sessions are helpful to decide what data has been considered as critical information that needs to be prioritized.
Modify the Rainbow Spreadsheet to synthesize data
The Rainbow Spreadsheet, introduced by Tomer Sharon, is a technique to collaboratively analyze user data. With this technique, pre-determined observations are created and used to guide synthesizing data across all users (each user is assigned with a color). The color and its distribution makes it easy to identify patterns. However, what if we don’t create pre-determined observations for research? Most of the time, specific research questions are used to guide data synthesis.
I adapt the Rainbow Spreadsheet to the situation when researchers don’t set pre-determined observations. In the modified Rainbow Spreadsheet, each row displays a specific research question. While each column (starting from the second column) still represents a user, colors are not simply assigned to each user. Instead, different colors are used to convey feelings across all users. For example, green indicates positive feelings (things work well); red indicates severe issues; yellow indicates issues with less severity. The data marked in different colors presents a clear view to identify how users reacted to different research questions, particularly where problems occurred. What’s more, different colors are used to illustrate different segments of users.
Listen to recordings to recall and extract meaningful user quotes
While transcripts contain detailed data, the written text sometimes doesn’t provide enough information to have a holistic understanding of the user responses. For example, The lack of the tone and stress used in talking about thoughts makes it difficult to fully understand users. Nevertheless, recordings can provide clarity. Listening to recordings allows me to recall the meaningful conversations and further to extract user quotes that support insights.
“For a qualitative study, if you go over each listening session transcript carefully, it’s magic how much better you understand that person’s thinking afterward. There is clarity that forms, and then patterns across different people coalesce from that clarity.”
– Indi Young
Recap
In analyzing and interpreting data, I’ve defined my North Star — to efficiently and effectively turn raw data into compelling insights — with the approach that contains the five techniques above. Hope my experience can help you get ideas about data analysis and interpretation in UX Research.
