Whereby or Lookback for remote user testing?

You plan to do remote user-testing or interviews, and you wonder which tooling to use… Here’s my view!

Enter the contestants!

Lookback

Whereby

So. Who’s the winner?

However…

  • Lookback needs a plugin and even then only works on Chrome. There’s no way to get it to work on Firefox or Apple Safari. That is a drawback! We need to tell all our research-participants they have to install Chrome and a plugin. Some of our participants are on locked-down computers and cannot install anything themselves. In comparison: Whereby runs on Firefox, Chrome and Safari — without any installation
  • Lookback forces participants to have a microphone. I like to call my participants using a regular phone call first, it’s reliable. During the call I setup the video connection. Sometimes it just won’t work and I use the phone for audio and an online tool only for the screenshare. With Lookback this approach is impossible. If a participant doesn’t have a microphone, it’s impossible to set up video or screenshare. I know this might sound like an ‘edge-case’, but for me this happens quite a lot. You’ll be happy to know that Whereby will work without camera- or microphone-access! Lookback Participate on smartphone is even worse: it won’t let you start a call without a headset
Image for post
Image for post
In Dutch it says: “Share your screen, choose what you want to share”, I selected “Your entire screen”. So why is the Share-button (bottom right) not active??? It’s a very confusing interaction-pattern in Google Chrome
  • Lookback screensharing is complicated. In my last usability-test, 9 out of 10 participants could not get through the three ‘easy-steps’ of Lookback without my help. This is not entirely Lookback’s fault: setting up screensharing in Chrome is convoluted, you need to click the screen you want to share even if you only have 1 screen, otherwise the button share screen won’t be activated. Whereby’s screensharing in Chrome is equally difficult to setup, but the saving grace here is that you can do it after setting up the rest of the call. Participants will first experience the happy “it works!”-feeling, before we struggle going through the screenshare-setup together
  • Lookback is expensive. Lookback costs $50 per month, per researcher. If you want to export notes and video, it costs $100 per month, per researcher. That’s crazy! Whereby is free. If you want to lock the room (a good idea), it costs $9 per month. If you want to record the call, there’s an added $5 per month.

There’s more annoyances (why do my participants need to enter an email-address to participate? Why can’t I chat or send a link to my participants during a call? Why is the flow of calling participants needlessly convoluted?Why the need for the“lookback-floating window” — participants continously lose or close it, etc.), but these are minor problems.

No more Lookback, ever?

What do you think?

So how do you user test with Whereby anyway?

  1. Recruit participants
    by screener or from a predefined list / participant pool
  2. Call participants by phone for planning a date/time
    I sometimes email them with a form to pick a date/time, but calling is much nicer: you immediately know if there are any problems. Always confirm through an email!
  3. Trial-run!
    A 2 minute call in a Whereby-room, trying out the participants’ connection, microphone and webcam. If necessary, we try out the screensharing as well.
  4. Run the test
    I wait 3 minutes to see if participants ‘show up’ in the Whereby-room. If not: I call them using a normal phone. If there are problems with their internet-connection or microphone, we keep the audio-connection through phone and run screenshare and/or video through Whereby. I record video through Quicktime or Lookback — the advantage with the latter is that the team can observe immediately

As always, thanks in advance for your feedback!

Image for post
Image for post
This photo is here because Medium would otherwise use my screenshot of Chrome as the image in the overview. Ugh. Anyway, it’s a nice photo. It kinda illustrates the idea of comparing things, right? — photo credits: myself on holiday in Venice

Why do you only mention Whereby? I think X is much better!

In short:

  • Zoom, WebEx and Teams: it’s possible to use it without installing an app, but they make it difficult to do. I find that a hurdle for my participants
  • Skype: you need an account and there are different types of Skype (business and personal) that can’t ‘talk’ to eachother (Skype for business is being phased out for Teams, though)
  • Google Meet: Google keeps changing the name and functionality of this product (Hangout? Google Talk?). Besides that, the interface is complicated

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store