Remote Research Platforms — A Head to Head Comparison
2020! What a year! Need I even say more?
Overnight, we were pushed to finding new ways of doing user research. While the world around us was sort of crumbling down, we at PeepalDesign resolved to take on the challenge.
Carrying out research remotely is no easy task considering that your participants and team mates are all in different locations with varying levels of internet bandwidth which could lead to unpredictable events such as
- increased dropout rates of participants
- increased cost due to need for recruiting sufficient backups
- communication gaps
- extended session times
- poor collaboration etc.
Furthermore, we needed to test out confidential prototypes of mobile apps, which meant we could not share them with the participant in advance.
We deep dived into a lot of tools to understand what could help us with our requirements and the following table gives a comparison between the tools we looked into across certain parameters:
Lookback is an easy to use tool for conducting moderated & unmoderated tests of mobile websites, apps or prototypes on desktops, iOS and android devices. The collaboration features enable you to invite teammates & clients to watch live sessions and can also allows them to take live time stamped notes and tag key insights.
- Participants would have to use a dedicated application — ‘Participate’ on their phone OR
- A chrome extension by the same name if testing on a laptop/desktop.
Challenges
- Moderator does not get a notification that the user has joined the meeting and is waiting for the former to start.
- There is no good categorisation for highlighting.
- Does not allow screen/remote control of the participants’ screen
2. Loop11
Loop11 can be used to test wireframes, prototypes, apps or websites. It is known to work well on prototypes especially those built using InVision, Axure and JustinMind.
It helps not just with online surveys but also in creating tasks and questionnaires for the users to perform. It offers a rich palette of features concentrated on the analysis especially quantitative research.
Challenges:
- Loop 11 does not allow for note taking or transcriptions on the tool
- It does not allow the researcher to share their screen and helps mainly in the scenario of testing apps or websites, prototypes when there is an integration of either of them with prototyping tools
- Loop11 does not allow screen/remote control of the moderators test phone by the participant
3. Validately
Validately is a tool that acts as a one stop solution for all aspects of research right from recruitment, scheduling of participants to getting automated reports post testing. Some of the highlights of this tool are that it can be used to test native apps as well, collaboration with multiple stakeholders and it’s ‘Insights’ feature.
Challenges:
- Validately does not allow screen/remote control of the moderators test phone by the participant
4. dscout
dscout is a video research platform that specializes in diary studies (capture in-context moments). It has 3 modules/aspects to it:
- Diary — conduct diary studies by designing missions for users to capture their experiences or moments on the mobile.
- Live — conduct remote research by recruiting from its own pool of participants called scouts.
- Recruit
It is most appropriate for Generative and foundational research.
Challenges:
- Dscout does not allow screen/remote control of the participants’ screen
- Not very suitable for Usability testing/ quantitative research
However, none of the listed tools helped us with our requirements that I’ve mentioned at the start of this post. All of the above tools come with its own limitations. We came up with our own workarounds to address the challenges. Stay tuned to read more about them.
If you are looking to conduct remote user research or testing you might like the following posts from research@PeepalDesign.
