UX Research in the time of COVID-19
This is a blog post I wrote more than a decade back, never thought it could become so relevant for the current times!
— — — — — — — — — — — — — — — -
Over the last few years, tightening travel budgets have led me to try remote modes of engaging with end-users in the US & Europe. My experience conducting remote usability tests (summative & formative) has been extremely positive.
- The think-aloud protocol works very well over the telephone. As the moderator, I feel that participants find it more natural to describe their thoughts and actions when the moderator is a voice on the telephone. When the moderator is the same room, they probably think that he can see what is happening and don’t voice their thoughts as readily.
- Even though we warn participants that performance might be slow because of the network and they should ignore it while giving satisfaction ratings, slow system or network performance tends to leave a negative impact on satisfaction rating. While this used to be a problem in the early days of remote usability tests, currently this is negated to a large extent by the almost ubiquitous availability of high bandwidth network connections and excellent conferencing applications. I believe that in the course of a few years this will completely disappear.
Since we focus on the usage of the software artefact during testing, a combination of screen sharing, webcam and audio conference helps us to get all the information we need to evaluate the interaction between participant & app. In my opinion, there is hardly any difference between remote & live testing. Given the effectiveness of remote testing, it should be used more widely than it is used now.
On the other hand, the results of my experiments with conducting remote user research is a completely different story. While you can get decent information during a ‘remote contextual interview’, a telephone conference with screen sharing can not substitute for the richness of the information that you get when you are present at the home or workplace of the user. For example, you miss out on all the contextual cues present like the physical environment and how it is laid out in relation to other people etc. If you ask me, conduct remote user research only as a last resort and even then try and bolster the data collected through these interviews with information gathered directly from users like photos, videos etc.
