Protecting and progressing services during COVID-19
Nile’s guide to remote research tools
Whether you’re in UX, innovation, or design, all your user research is remote research now. What are the best tools to engage with customers remotely?
The ongoing pandemic is changing the way people value and interact with products and services.
Think about a couple of examples. Sainsbury’s online grocery slot booking system used to offer convenience and predictability at a low cost. Now, that little widget is the thing that decides whether we get to eat tomorrow, or need to wait for a slot later in the week.
Or take WhatApp’s video call function as an example. My parents used it as a gimmicky way to do silly video calls with my infant sons. Now it’s the only way they’re able to see and maintain a relationship with their grandchildren.
Like it or not, your business proposition has changed
Sainsburys has become an essential service (and to their credit they’re moving fast to support vulnerable users effectively). WhatsApp is keeping families together. As customer context changes, the role played by the services you provide changes too.
Here at Nile, we’re forecasting that many of these propositions will remain changed for at least the next 12 months. In some cases, perhaps permanently.
If you own a service or product, it’s time to check in. Now more than ever is the time to be talking with customers to understand their needs.
But with so many teams new to remote research, many of our clients are asking for recommendations about effective remote tools to engage users and customers.
But first — lots of teams are slowing down their customer engagement at the moment. And that’s healthy.
That’s a good response to the crisis, for both research teams and potential participants.
While customer engagement is arguably more important than ever, we’re advising our clients to consider a few key questions before they embark on a period of customer testing — even something as seemingly uncontroversial as UX testing:
- Is it necessary?
- It is necessary now?
- Should we or can we do it differently?
- How can we minimise the impact on our participants?
- How can we care for ourselves? (We may hear traumatic stories.)
Assuming that your research is necessary and you’ve considered alternatives, mitigations and caring strategies, here’s Nile’s guide to online panels and communities, pulled from our own experience over the last few years of running remote research.
Working with online panels
Online panels are essentially a quantitative survey resource. We regularly use Panelbase, FlexMR and YouGov to provide quant support on many of our projects.
Providers like these administer large panels, often with between 1,000–10,000+ participants. They’ve all been recruited and retained by the panel provider, amd can be called on to respond to questions you need answers to.
Panels are great in helping us answer questions like:
- How big is the problem?
- What are people doing and how are they doing it?
- How many people does the problem affect?
- How many people think x?
- How many people like x, y, z?
- How confident are we that x?
Providers often offer different levels of service — from the bare bones of hiring the panel and platform, through to compiling surveys for you and crunching the numbers at the other end.
There are specific types of questions which work well with panels — these are your typical constrained survey questions — multiple choices, rating scales, radio button selections, short/long free text answers, etc. Many platforms allow you to embed links, images, audio or video, giving greater flexibility and making the survey experience more interactive and engaging for participants.
Online panels are great for when you need…
- To dive into people’s attitudes and (current) behaviours. But beware — you need to pose fairly simple questions because online panels cannot be moderated — there’s no way for you to dive in and help out if respondents struggle with something.
- ‘Proof’ or validation. If you need to validate or quantify findings from smaller-scale qualitative work, using a large panel (ideally representative of the population you’re working with) can give you the certainty or proportionality you need.
- Quick turnaround. Results can be returned in a matter of days. Some services will even run data analysis for you. Depending on your team’s skill level, time and budget you can choose a package that suits you.
But beware: Panel members are usually members of more than one panel. ‘Survey fatigue’ can set in. We often notice that the longer an individual is a member of a panel, the less engaged they become and so offer less value.
At Nile we rarely rely on a single data source on our projects. We’re always looking to layer insights to produce a 360° view, but also to minimise any bias or effect of a particular research method.
Working with online communities
Communities usually have a smaller number of participants (between 30–100 people) and typically offer more varied types of tasks and activities. We tend to use DScout and Together (Go Further) in our day-to-day work.
Communities offer great value and flexibility. You not only have a large variety of activities to choose from, but can also set up tasks as either private or public (to the group). This allows you to cater for those sensitive topics or personal questions, but also benefit from group dynamics where needed — such as idea building and iteration — all within the same study.
Communities are great when…
- You need to build a picture of a particular group; understand their behaviours, motivations, triggers and barriers. Following and interacting with the same group of participants over a set period of time allows you to gather rich insight and data about them and their environment. Having this longer view also allows you to develop your understanding of the deeper psychological motivations which underpin people’s behaviours that you are unlikely to get from one-off engagements such as surveys.
- You are exploring a complex subject. Communities lend themselves to Discovery-type projects, as well as during the Design and Explore phases of a project. That’s because, with good moderation, online communities can handle complex or multi-faceted subjects that might otherwise baffle a panel survey. Participants can build their own knowledge and understanding of the topic or concept being investigated. However, this means that a moderator needs to be available for the duration of the study to ask questions, respond to members, collect and tag data and steer the study in the direction it needs to go. As a result, communities are resource-heavy in comparison to panels.
And remember:
If you’re engaging with a community, you have to engage. This means researchers need to be skilled, personable, and available. You’ll get best results by regularly probing the subject, providing feedback and encouraging participants to interact amongst themselves.
Iterate and build. Even though your community may only be running for 1–2 weeks, you should think about how the exercises build from one day to the next and incorporate previous responses and insights. On the flip-side of this day-by-day planning, it is also important to leave some space for serendipitous exploration and reflection time.
Other variations and tools to consider
Diary studies
Diary studies allow you to build deep empathy with your customers, and understand how they behave in context, and at the time of use. If designed properly, they can offer a slick participant experience and provide extremely rich and valuable insights. Participants self-report over a set period of time. On certain platforms a diary study can be incorporated in to a larger community study. At Nile, we regularly use DScout or Together.
They’re great for providing the essential context you need when exploring human behaviour. This is crucial when you are exploring particular experiences susceptible to environmental factors and influences (and, arguably, most are!). Diary studies can open up a deep understanding of long term habits, attitudes and motivations, and so are often an effective way to identify unserved needs or explore opportunity spaces.
Pulse or Rapid quant survey
Pulse / Rapid Quant surveys are the quickest way to capture ‘signals’ or quantify one or two simple attitudes or preferences. We’ve used OnePulse and StreetBees to great effect in the past.
These platforms coordinate and manage a large set of mobile participants, poised and ready to answer questions almost immediately (often via a mobile app). You can decide the number of questions to ask, the number of participants to engage, and the demographics you’re looking to work with.
When to use rapid quant surveys — These are great for getting an early sense check on themes or to answer simple and straightforward questions. Also they’re fun to watch — seeing the number of completions tick upwards after launch is very satisfying.
Some notes on choosing the right tool
Before choosing the tool, first, think about the type of project you are doing, what capabilities you have, your budget and timescales. These are all going to play a part in choosing the right option
For example…
Let’s suppose you have limited experience running remote user research, and want a one-off validation of a piece of qualitative work. You might consider using a partner (like Nile HQ #plug) or a panel provider who can manage the process end to end — taking your objectives, and progressing the whole endeavour, from survey script writing all the way through to the analysis of the data.
Or perhaps this project is the start of an ongoing effort to carry out remote user research. In that case, you might consider familiarising your team with an online community. This will help you to engage with the same representative audience repeatedly, building insight on insight over a period of time.
Finally, maybe you’re just looking for a quick temperature check on something, or a sense check on a theme or hypothesis for next to no money. A rapid quant survey is the answer in this case.
Do you have any other recommendations or experience with providers you could share? Drop us a line in the comments below. Want to know more about running remote research, or need to talk to an expert? Get in touch with the team at Nile HQ.
Credit to Dr Alexa Haynes and Robyn Johnston for sharing their expertise on this article.
