Dear Researchers, Don’t Take User Behavior for Granted
At one research presentation, one researcher said to me “I don’t think your report is true Tya, because I don’t behave the same way as you describe”. Startled, I thought: wow what an intense question to end the session. Here’s what I learned from the question…
Our confidence to share our research is built on repeated actions when we conduct every phase of research. I believe that the more we practice, the more we get insights about what’s good or what still needs improvement. Also, as we get more diverse projects, we gather more experience and wisdom in treating and reporting our data. But in between those repeated work, are we consciously evaluating our research actions or are we being ignorant of our actions?
The action that I’m trying to highlight here is how we react towards our work and others, just like the question I got. Following the event, I was reminded about “taking behavior for granted” from a qualitative research textbook. Taking behavior for granted (or in this case, user data) means as a researcher we ignore or even devalue a behavior because we think it is not important to our project, or even worse to us personally.
Being ignorant about how we take behavior for granted can lower the quality of our work output because we create inaccurate conclusions, jeopardizing our report validity, and further affecting relationship and trust from other roles. So how can we be more aware and avoid taking behavior for granted? I propose 4 things to remember for researchers:
- Aware of our own bias & misunderstanding on qualitative research
“We’re prone to bias”, we hear it repeatedly but do we really get what it means? In my case, I think the researcher forgot that human is not the best observer for ourselves. Also, the researcher portrayed a dichotomic view on behavior: right or wrong, true or false. But this argument happened based on one reason: the behavior described in my report was not similar to their behavior. Needless to say, this is an incomplete view of qualitative research: it’s not about one single truth or generalization. As a researcher, we need to be aware of our bias. Let go of the thought to prove our forecast is true.
2. Know the difference: raw data vs analyzed data
The reported data presented at the end of the research has gone through a thorough analysis. It means, researchers usually look for repeated patterns from the data until they reach data saturation (no new information found in the data despite additional data) or theoretic closure. With these reasons on analysis, researchers need to be really careful in comparing their own experience (which is a raw data) with the analyzed data. Again, it’s not about proving one behavior is right or wrong, but to get proper context of the phenomena.
3. Have the right point of view: check for validity through triangulation
There are times when I found that researchers reported user behavior in controversial ways: people are opportunist, disloyal, or in product context: people don’t like this concept & feature but not suggesting any alternative. These acts resulted in stakeholders being confused and panicked, thus created tension between us and them.
When we face this kind of data, make sure to check it again: is it contradicting another fact (perhaps concept A is widely accepted in another context)? Are people really opportunist or it’s just how people decide between different options offered to them? We need the right point of view when analyzing these data. So check your process to make sure your result is valid: are you doing the analysis right? Are you taking meaning from word for word to everything? Are you biased and projecting your own beliefs? Have you considered another way to see the data, like when you are asking a cost-related question to sellers? Perhaps, to build a better result and get more experience, researcher needs to involve another researcher (investigator triangulation) to make sure of the validity is properly analysed.
4. Document data properly, be less reliant on memory
Some researchers consider that not documenting data properly is normal. Whether it’s the sound recording, direct quote, observation notes, or scribbles on paper-prototype. Then, they go straight to analysis based on what they can recall. Let’s call this overconfidence on the ability to memorize, forgetting the fact that our memory fades over time. Not documenting increases the probability of missing important point, not getting the whole picture on behavior, missing implicit meanings, and other possible complications. When you take extra effort to document data properly, then laid out the data from different sources, you can gain depth of the result in your report.
Researchers, if you remember how hard it is to recruit specific participants, to wait through uncertainty whether the participants will attend your session, you know that getting the data is a tough process. When you feel unsure, you can always create collaboration with other experienced researchers or create a co-analysis session. And for other roles that work closely with us, always confirm your conclusions to the researcher, do raise your concern to other researchers if you think something is not right, be critical upon accepting our report, and collaborate through the process.
Let’s be more aware of taking behavior for granted.
Thanks to Iqbal Mabbit Satria and Syafrizal Wardhana!
