Some (Significant) Advantages of Doing Qualitative Design Research Remotely
After spending over twenty years in the field of Experience Design and Design Research, doing much of my work remotely, I’ve been thinking about how I can best be of service to others at this critical juncture (as I’m sure many of you are). The times have found us needing to step up our remote work skills and reevaluate our strategies for getting research done —while still being confident in the validity of the qualitative data gathered.
Remote research can not only work just as well as in-person research, but can actually be an advantage in many cases.
If you’re feeling like everything you’ve been told about doing design and design research (UX, HCD, etc.) is that you can’t be as confident in your findings when doing research remotely, I do have some great news for you: remote research can not only work just as well as in-person research, but can actually be an advantage in many cases. Yes, there are some advantages sometimes of doing research in person (I’ll acknowledge those at the end of this post). But the current conversations around this issue tend to jump to assumptions and conclusions that remote research is inferior, when it’s actually proven itself to be quite advantageous to me and others, beyond just the convenience and cost-savings factor.
This article won’t focus on remote research tools, by the way — most of you probably know there are many great tools out there for online design research and collaboration, and you can read all about them in other posts and webinars. I’ll briefly mention a couple personal favorites: Calendly for managing scheduling nightmares across timezones, and Zoom for stability, relative ease of use, and its transcripting feature (business edition — just beware that non-native English speech accents tend not to transcribe so well).
So what are these remote research advantages I am claiming?
The Basics: Global Input + Saving Money
There are very few products and services that are hyper-local these days, and I work mostly on projects with a multi-national user base, so doing remote research has practically been a no-brainer for my projects and clients. Most organizations simply do not have the budget to fly researchers around the globe to gather qualitative data. The only question you should have is whether your particular project requires observation to have any validity at all, and for most projects, I think that is not the case. (And even with observation, there are ways to get creative with working remotely, as I mention at the end).
More Inclusion of Diverse Participants (if you’re targeting a wide-ranging user base)
While this point can be argued a few ways, I believe remote research can generally be more inclusive when you have a wide-ranging target user base. For example, you will probably find it easier to recruit differently-abled people who have mobility or chronic health issues, people in rural settings, and people with very busy schedules, such as parents. Where online research can be a limiting factor is for folks who might have connectivity, vision, cognitive, or other challenges when it comes to using online tools — or if they are simply not comfortable with the technology. Fortunately, Internet infrastructure is getting better everywhere, the conferencing tools are getting more commonplace and easier to use, and you can prepare a brief tutorial of them ahead of time, anticipating usability issues some may encounter. I find it’s always good to get participants prepared as much as possible ahead of time with necessary application or plug-in downloads, as well as creating a user name/ account for themselves if needed. And don’t forget there is always the good old-fashioned phone for simple interviews or possibly conference/ group calls, which is still probably accessible to many more than Internet.
Increased Comfort Level of Participants
This one tends to be a surprise to even more experienced researchers. But yes, especially for some topics where participants are could be feeling vulnerable (and even discussions around technology can make some people feel vulnerable if they feel they are being tested — but in general, I’m talking about more emotional topics), it can actually be an advantage to not be the same room with them. And I’m not the only experienced researcher to have noted this.
Why would this be? I have to speculate. But first, doing your research in an online conferencing format means your participants will choose their own comfortable environment — their own homes or offices, most likely, where they feel most in control, as opposed to artificial settings like workshop/ conference rooms or test labs.
The second reason is more interesting. It seems that putting some distance between yourself and the participants can actually help them open up about issues where you’re asking them to reveal their vulnerability — where they may feel insecure, embarrassed, angry, scared, or sad, and I tend to think these emotions can be more difficult to share in person where there can be a lot of sensory overload in the moment , being with another person or people. Think of professional therapists who take a long time to achieve a level of comfort and trust with their clients. Design researchers don’t have their time, nor the expertise (though I believe demeanor and moderation skills are critical, no matter the format, in design research).
Putting some distance between yourself and the participants can actually help them open up about issues where you’re asking them to reveal their vulnerability — where they may feel insecure, embarrassed, angry, scared, or sad.
Still skeptical? Think of the success of platforms like 7 Cups of Tea, text, chat or phone support for personal crises, such as the suicide hotline. And while one can argue there may be limitations to the assistance that can be provided through these formats, the folks seeking them out still show up en masse for them, and not at their therapists’ offices.
I recently learned about this comfort advantage whie exploring emotional topics for a large design research project for a United Nations agency’s Human Resources department. I surveyed and conducted focus groups from a pool of 4,000 staff members around the world, studying their perceptions of experiences of a challenging policy and its corresponding process. Our call for participation in the qual research was met with an extremely strong response; staff members wanted their voices heard. This meant I had to keep one-on-one interviews to a minimum, for scope reasons — but still ensure participants were as comfortable as possible discussing, in a group format, this controversial policy that had immense impact on their personal lives and careers. I traveled to four cities to conduct some in-person focus groups, but doing remote groups was a necessity with staff in over 190 locations around the globe.
What did I find? More than in the in-person groups, many online participants thanked me for creating a “safe space” to share. Many took me up on my offer to make up a fictional name for the session and keep their video turned off. I will add, though I’m aware of the research on the importance of facial and gestural expressions, I am now somewhat skeptical about it. Some faces are naturally more expressive than others, but I personally feel I find less variance and a better read combining an analysis of words spoken with tone of voice. So audio I found to key, but video — to me, not so much. Anyway, relatively speaking, the online groups were better attended.
Regarding confidentiality and data security for online sessions, in my call for participation as well as my session reminder/ prep communications, I had to use participants’ e-mail addresses, so I personally knew each participant’s actual name. But I firmly committed to: keeping participant identities confidential from even my colleagues in HR; keeping staff names out of my final report; and destroying all session recordings and transcripts after I downloaded them from cloud storage. I pledged all of this to the participant pool even prior to recruitment. Of course, I told them I welcomed tough, frank feedback, and expected tolerance of other’s viewpoints. Additionally, mixing staff from different time zones helped with providing the best chance of complete anonymity in the groups — a few participants did recognize one another, but most did not — and I didn’t perceive any concern over being recognized because of my other commitments to confidentiality in performing this research. Mixing time zones is just something you can’t do on-site! (And the regional interactions provided additional insights as well.)
Easier to Moderate Group Formats
Another possible surprise: while I wouldn’t say there was a significant difference between the in-person and remote groups in terms of participant behaviors, the in-person groups tended a bit more toward hot debates, and sometimes, a heightened contentiousness; while online, people made their points very firmly, but tended more toward diplomacy — they also debated a bit less. Both types of group reactions were valuable. But the online discussions, while consisting of some very tough feedback — were easier to moderate and keep from going down a rabbit hole. In person, I occasionally had to play referee to move the discussion along when it was clear the participants wanted to continue to debate the same point again and again.
And why would this be? I think the pace of online group sessions is slower and more deliberate by default; people must talk one at a time to be understood at all, whereas in person, you will deal more with potential challenges of people talking over one another, and possibly dominating the group. What ended up happening in my online sessions is that I would often “call on” each participant to give their reaction or response to my question or prompt one by one, and that person would often build on what had been said prior to them. In person, it was much more loose and spontaneous in terms of who spoke. A take-away here: if you think hot debates between participants would valuable to your research, ideally, you may want to at least augment your online research with in-person groups — or, use your creativity to determine how to encourage deeper debating in a session online.
Easier (and More Inclusive) for Researchers
It probably goes without saying that while travel is something many researchers enjoy, it is draining. Multiply the fatigue factor by at least ten if you are investigating a highly emotional, complex, or contentious topic. Many of us have highly sensitive personalities (I myself am Myers-Briggs INFJ, the “counselor personality type” so I am often told my sessions “feel like therapy,” participants comment on feeling glad to get “things off their chests” ( even though I’m definitely not a therapist, nor do I play one in my work!). I truly love to travel, but also do benefit from “hermitting” while researching — it helps me manage my empathy burn-out for social impact research. While we shouldn’t choose methodologies for our own comfort first, we must realistically manage our burn-out, so it’s just another factor to consider — along with inclusivity for researchers with mobility challenges.
Reasons Not to Research Remotely
While I believe great quality research is possible in the large majority of remote research cases, there are instances where it may not be inappropriate, such as:
When observation is important — although do consider if webcam or someone taking you for video tour can help with that data gathering (ethically and with consent, of course)
When the participants demand you be there. I have had participants who simply just don’t “do online” (even if they could); either you show up in person or they won’t even schedule a meeting with you. Hopefully these sorts of mindsets are changing now, but you may (still) also run into people not as comfortable with remote sessions, for various reasons.
If you feel it would be unethical for any reason. While there are no hard and fast rules here, I might not perform research on topics that have minors, for example, exposing their vulnerabilites in online sessions, unless you can guarantee they have a support system in person for session after-care, or even an adult guardian or counselor in the room with them.
If there are language or connectivity barriers. If you don’t speak the participant’s native language, it may be easier to have a translator participate in person, though this would be entirely possible for online research as well, if you prepare and hire correctly. More often, I’ve run into connectivity issues hampering online research that can be difficult to anticipate ahead of time, so always plan for how you’ll do your session without video and screen sharing — just in case.
Remote Research vs. Remote Research Assistance
Some practitioners recommend coaching others in the target localities to conduct the moderation in-person, rather than traveling there themselves, in a pinch. I do think this strategy could potentially work — but only with appropriate screening for natural moderation “talent”/ demeanor (yes, there is such a thing!) and investment in training. I’ve personally witnessed moderators who are very smart people — at the “top of the top tech” companies, and very invested in the idea of design research — actually fail in their sessions because of their demeanor and not being able to make participants feel at ease. So do be honest with all involved here, and beware — if you get a moderator who is intimidating or leads in any way, you greatly risk wasting time and money.
In any case, with COVID-19 being the current issue of concern, the suggestion of shifting burden of in-person contact to local resources feels irrelevant and unethical, unless there is certainty of practically no infection risk there (and unfortunately, this looks to be less and less the case anywhere).
In Conclusion
A lot of the focus here has been about the comfort of your participants, for good reason. And if you’re unsure of their comfort levels in participating in online research, after considering all of the above, just ask them. If you’re a consultant, you could make this part of phase 1 in your proposal, with the rest of the phases TBD based on participant response, and give a preview of the proposed remote methodology.
Finally, I well know, after twenty years as an independent consultant, that the isolation that comes with remote work can be challenging. But you will adjust (and if you’re an introvert, you will naturally rise to the occasion)! A final note about remote collaboration with colleagues — many in tech prefer it! I had a wonderful remote working relationship with a product manager at a nonprofit in San Francisco while I was in Boston for years. Yes, some people find it easier to manage their own work without us around (go figure!).
While this is a very tough time, I feel grateful that the technology exists to make remote work possible — the disruption (economic, psychological) is tough enough. My sincere best wishes to all—please reach out if you have questions.
