Remote user research — the actionable checklist

Conducting remote user research — how do you do that? User research is all about the users. But what if you can’t get in touch with them on-site?

Not only a pandemic, but also geographical conditions or a lack of time require us to conduct user research remotely. While we simply move our meetings to Zoom, Google Meet, Teams, or Skype, we also need to adapt the user research process to new conditions:

Even before Covid, we (Jonas and Dominic) were experts in the field of user research. With the transition to remote work, we learned a lot of new things we would like to share with you as a guideline for remote user research.

Our guideline has the following chapters:

Chapters

▶️ 0 Onboarding

🗓️ 1 Planning remote user research

🧑‍ 2 Participant recruiting for remote user research

📝 3 Pre-interview preparation for remote user research

👨‍💻 4 Conducting the remote interview

📊 5 Post-processing

💾 6 Case closing

(0 Gain basic understanding)

This chapter may be only relevant if you conduct remote user research for the first time.

0.1 Carry out an internal playtest

For an internal playtest, the aim is to test remote user research itself to see how it works. To do this, you create a so-called “fake study”. Don’t worry about choosing the perfect test user: just ask a colleague or another person within your company.

The goal of an internal playtest is to identify where problems occur in remote research and how to fix them before starting remote research in practice.

0.2 Participate in community testings

If you are new to user research, we recommend participating in a community testing session. Why?

• 👥 Roleplay: Put yourself in the moderator’s or a participant’s shoes. Especially having been a moderator once helps to do a better job later on.
• 💭 Feedback: During testing sessions, you can ask for two kinds of feedback: First, feedback on your moderator qualities. And second, feedback on the test object you bring to the session.
• 🗪 Exchange: Besides, you can always learn something new by exchanging ideas with others.

For the German area, we recommend the community testing event “Usability Testessen”.

0.3 Review existing recordings

To prepare for a user testing session, it’s worth revisiting old recordings. On the one hand, you see what to improve for the future in a methodical sense. On the other hand, you learn how to pose questions and what to dismiss to get the insights you want.

0.4 Register with a panel

Another way to gain insight into the world of remote user testing is to register with a panel yourself. We know this is time-consuming. But to have been a test user in a real testing session helps you to understand your own test users way better.

1 Planning remote user research

1.1 Clarify the expectations

To get the desired results from your user test, ask yourselves the following questions:

• What kind of insights would you like to gain from the study?
• Are there specific hypotheses you want to test or is your study rather explorative?
• Do you use a lean concept or is it an in-depth study?

1.2 Define your study characteristics

What should your testing session look like? Define that by the following criteria:

• 🎯 Focus: general UX or specific usability topics?
• 📊 Character: qualitative or quantitative?
• 🔍 Methods: e.g. user interviews, classical usability testing, usability test, remote eye-tracking…
• 👨‍🏫 Moderation: moderated or unmoderated?
• 📐Measuring: Do you use scales for analyzing the results? Which ones?

1.3 Choose your tools

Remote research thrives on digital collaboration. As for the tools, you will need a minimal set. This includes:

• Communication tools (phone, conferencing tool, and remote user research tools if necessary)
• Supportive software (e.g., for collaboration with your assistants). For taking notes, we recommend tools as Miro or Google Docs.

Regarding the tools, you need to consider the following:

• ✔️ Availability of the necessary functions (e.g., recording)
• ✔️ The tools’ GDPR compliance
• ✔️ License availabilities
• ✔️ Expiring trial periods
• ✔️ Possibilities to test a tool before the purchase

1.4 Write a script

Let’s move on to our script for the testing sessions. In this step, you plan the tasks. They define what your test users will have to do while you observe their behavior.

❗️ Set the tasks openly: When you give your test users instructions, you tend to guide them through the session. Thus, this may falsify the results.

1.5 Make a backup plan

We’re almost done planning! But there’s still one very important thing: When things go wrong, a testing session is only as good as its backup plan. Therefore, such a plan should contain:

• 🧑‍ Support person: During a testing session, you may not have the time to fix your technical issues yourself. Therefore, we recommend asking a colleague in advance
• 💫 Reliable internet connection: Besides wifi, a connection via LAN-cable or the possibility to open a hotspot can save your remote testing session.
• 💻 Alternative devices: Have emergency equipment at hand, to avoid complete technology failure. This means, for example, to have a second computer you can use.

2 Participant recruiting for remote user research

2.1 Use a scheduling tool

Use scheduling tools, such as Doodle or calendly, to make appointments for testing sessions with your test users. This way, users can schedule their appointments themselves (within a fixed period).

2.2 Collect phone numbers

It is advisable to have your test users’ phone numbers. That way you can call them if they are not online at their scheduled time. Also, you can contact them for technical issues like a failing internet connection and help to find solutions.

2.3. Send a briefing in advance

You can save a lot of time during the individual testing sessions by sending the test users a briefing by email in advance. This way you spend less time explaining and the test users can ask their questions right at the beginning.

2.4. Include buffers

A golden rule of user testing is to include buffer time. Unpredictable things may happen that might lead to a delay. We suggest including at least 15 minutes, better 30 minutes between two test users.

A buffer is also useful to review your notes from the previous session or to prepare yourself for the next.

3 Pre-interview preparation for remote user research

3.1. Check in with the participant

Though you’ve already considered buffer time, there is another tip to save time: Call your participants one day before the testing session. This way, you get the chance to address the following points:

• 💫 Internet connection: Is it good enough for the remote testing session?
• 🔊 Sound and video quality
• 💻 Software: Does everything work?
• ❓ Questions: Clarify the participant’s questions or FAQ (such as details on the consent or who will be the interviewer)

3.2. Carry out a dry run

Once the study has been planned, you should carry out another dry run. As with the playtest recommended at the beginning, the participant can be anyone.

3.3. Check the consent

Data privacy should have a high priority. Therefore, check whether this list applies to your consent form:

• ✔️ Correct date
• ✔️ Correct general information (used software, use of the collected data, and data storage period)
• ✔️ Correct information about the test user
• ✔️ Especially important for remote research: Can the form be digitally signed?

3.4 Block distractions

During a testing session, your test users should be undisturbed. This means that you should turn off everything distracting. This includes:

• ✔️ Email notifications
• ✔️ Pop-ups
• ✔️ And yes, even your smartphone (or at least mute it)

Distractions can also occur in the analog world, for example, when people enter the room without an announcement. A “please do not disturb” sign will help.

4 Conducting the remote interview

4.1. Turn your camera on, be a proper professional

While conducting remote user interviews, you should turn on your webcam, too.

Otherwise, the users may feel that they are being observed and treated with less respect

4.2 Clarify consent

Consent is king! Again, ask for permission to record the session!

❗️ Keep the legal framework in mind. Also note that when you conduct studies with an international background, the legal conditions might be different!

4.3. Build trust

Remote user research doesn’t involve a face-to-face situation as you have it in a lab. Therefore, it may be difficult for the user to build trust.

• Point out: “We’re testing the product. We’re not testing you.”
• Encourage the user to ask about technical issues or unclear instructions.

4.4. Avoid privacy issues

Attention!

When a testing session takes place at the test persons’ homes, they use their own devices. Thereby, someone may accidentally have things on the screen that should not be seen.

Take a note of when such a situation occurs so that you can delete it from the recording later.

4.5 Invite stakeholder(s)

Sometimes your stakeholders are present during a testing session. In this case, it is important not to scare away your testers.

• 💬 Communicate transparently! Explain to your test users that, for example, the customer for whom you are conducting the study will also be present at the test session.
• 🚫 However, avoid including the stakeholder with a camera. That can be distracting and intimidating.

4.6. Record the session

Before you start recording, announce it. Again, it is important to ask for the user’s consent.

Also mention what is going to happen with the recording, its (short) storage duration, and deleting date.

4.7 Questionnaire

You’re conducting unmoderated user research using a survey? Then we’ve got an extra tip for you: Let your users talk while filling out NPS, SUS, or Attrak Diff. This way, you learn more about the “why” and not only the “what”.

5 Post-processing

5.1 Incentives

Incentives are usually handled by the panel through which the test users were invited.

Otherwise, you can simply send gift cards or make a bank transfer.

5.2 Collect notes

Organize and collect your notes. Pay special attention to implicit information.

5.3 Handle data

When working with the collected data, make sure to cut out or anonymize PII data (= personally identifiable information; see 4.4).

5.4 Analyze data

Time to analyze your data. Have you ever applied the principles of Atomic UX research to your research process? If not, we’d love to tell you more about it.

5.5. Communicate your findings

Share what you found out with your colleagues! You can do this by hosting a video session or a campfire session online.

6 Case closing

6.1. Sustainable storage

When storing your data, make sure:

• 🔐 You can easily access your data and findings later on.
• 📚 You store your files well-structured.

To profit in the long run, store your data in a repository. If you don’t have one yet — check out this article on why you should think about implementing one.

6.2. Document new hypotheses

User research is never finished!

Write down what you’ve learned and derive new hypotheses from it which you can explore in your next remote user research session.