Making sense of qualitative data
Research is formalized curiosity. It is poking and prying with a purpose — Zora Neale Hurston
In the last year, I have conducted many research studies on different topics. Through this, I met interesting people, improved my listening and interviewing skills, informed product development, and experimented with several methods of data collection, analysis and synthesis. Like other fields, research is an iterative process, you improve on the execution and reporting as you engage in more studies.
Several times, after a conversation, interviewees have said, “I hope this was helpful for your work’’ or asks,’’ So what next after this? What should I expect? What will you do with this information?”. While in most cases I usually have taken them through the consent and brief of the project, I find these questions grounding because they take me back to the why of the research. The journey doesn’t end with closing the discussion, there’s a need for rigorous analysis, synthesis, and action on what we learn every day.
There are several guides online on how to conduct UX research or any research in general. From prepping, getting to the field, analyzing, and reporting. But not much is mentioned about the nonlinear flow this entails. In a previous post, I shared briefly about some things that can and have actually gone wrong during research.
The messy middle, death by data
The amount of data to analyze really depends on the project. For larger projects that require many interviewees, it’s easy to drown in information once the research is complete. I have been there several times. Let’s assume you’ve had a successful run with a qualitative research study, the planning went well, you recruited the right participants and carried out the research successfully. Now you are sitting in piles of information in the form of videos, audio, diary entries, transcriptions, notes, pictures, and other reference materials. You are possibly thinking of the complex stories you need to highlight, best ways to represent the data and communicate the rich results.
There are several methods of analyzing qualitative research, a personal favorite being thematic analysis. Here, the researchers examine the data identifying common themes and draw patterns from this. This approach can be used for explorations where no themes have been identified yet or in a deductive study where one has some preconceived themes or has created a framework to use. One thing to note is themes don’t just emerge, as a researcher it’s your active role to identify them, figure out patterns in the data and keep iterating. Analyzing research findings is very involving, requires iterations and creativity on the go, it gets better with time. To get the best out of the data, here are some tips:
Sit with the data longer.
Often, we go into research with preconceived assumptions and personal biases. We might be tempted to jump at the first insights that prove or disprove our assumptions and forget those that challenge the research questions or are ambiguous. When you sit with the data longer, you can have a better perspective. Study the data, move away then come back, if possible a few days later. You might be surprised at what you learn from it. Over time, I have learned to allocate more time to the analysis step, something I had earlier assumed would be quick and easy, it’s never easy but worth it.
Don’t be afraid of ballooning themes.
Once you are familiar with the data, start coding. Coding is assigning tags or themes. Start the initial coding, either by using an existing framework or creating one as you go. Allow yourself to go broad and then narrow. Identify the patterns as you go, feel free to iterate on the codes, review, and redefine.
Have a collaborator who helps in bringing out different perspectives.
Having a collaborator helps eliminate bias and guides you towards seeing something in the data that you might have not seen. Through discussions and comparing notes, new themes and patterns might emerge. It also makes the analysis more fun. If you do not have a project collaborator, take time talking about what you are finding out with someone else, anyone. Sharing what you are learning creates more clarity.
You might want to use a tool for analysis
Most of the work on analysis involves in-depth thinking, the tool is there to help you stay organized. Always have an idea of what you want to do with the data in the end. I currently combine both online and offline tools in analysis. Using an analysis software does not sit in for pen, paper, printed notes, post-it notes, etc. You can have both worlds. The tool is just that, a tool, it doesn’t create the themes magically. While it helps keep you close to your data, analysis requires creativity, it is an intellectually demanding task.
My experience using Dovetail
One tool that has been handy for analysis is dovetail. It’s great for collaboration, giving life to the data through visuals and great user experience, and helping put all the data in one repository for easier referencing. Not only does it make surfacing insights easier from raw data but also helps in backing up insights with data points e.g. easy reference to quotes. With dovetail, you can organize research studies into projects. Inside a project, you can create boards and groups based on your data, then proceed to add tags to the data as you code. Once coding is done, you can easily view highlights by sorting the data based on the groups and tags, create charts, draw insights from the data and invite stakeholders who might want to follow the study closely.
When using dovetail:
- I noticed the transcription tool on the app was not very accurate hence opted to upload already transcribed data. Dovetail has a transcription option where you can upload a video or audio file, if you are comfortable with the transcription, then no need to upload transcribed notes.
- When coding, assign a tag to rate the quotes. Including quotes in research insights offer a chance for others to hear the participants. If you come across a quote that you like and would like to use in your report, you can add a rating to it e.g. a scale of 1–5 with 5 being for the best quotes that you definitely want to have in your report.
- When coding and creating themes, be on the lookout not only for what is within your hypothesis but for what contradicts your early assumptions. Create new tags as you go, you can then revisit these contradictions later even if you are following a thematic approach. There is a lot to learn from them too.
Keen to note that analysis is fraught with missteps, always leave room for some change. It’s far from a tick box exercise, leave the door open and let the data teach you, is that not why you conducted the study?
Feel free to share how your research analysis and synthesis have been. What tool are you using? What have been the highs and the lows?
