Maintaining collaborative user research during lockdown: what we’ve learnt and what we’ll keep doing
Head of Inclusive Design, Jeanette Clement, talks through what it’s been like running remote research during a pandemic — from technical difficulties to triumphs.
It’s fair to say that pre-lockdown our team relied heavily on lab-based research, whether interviews or usability tests, we automatically brought users into our lab to run the research. The only research we did fully remotely were activities like tree-tests or card sorting. Things that were significantly easier to run with online tools.
We were already using collaborative tools like Miro and Notion for some workshops and reporting, but we still relied heavily on Post-its and Sharpies. Being at least familiar with these digital tools was helpful as we were soon to become utterly dependant on them when our world — like everyone’s — got turned upside down with the COVID-19 lockdown.
There are four main collaborative activities that are indispensable to our teams whether digital-first or not:
- Generating research questions
- Preparing for research
- Running research
- Analysis
All of these activities had to continue somehow, despite our researchers and participants being in their own homes in locations across the UK. Without them, we simply wouldn’t know whether we were building the right things for our 34m+ customers.
Generating research questions
Before we start doing any research it’s important that we understand what we want to learn. This activity helps the whole team contribute to the research questions. Research questions are the things that the team want to learn from research (not to be confused with the questions that you ask users during a research session).
Templates in Miro are great for setting up activities quickly and ensuring the process is consistent across multiple squads.
The Miro board in this template is divided into three areas: problem statement, research objectives, and priorities.
We find it is best to get the team to pre-populate the problem statement before the session, there is guidance on the board of what they need to do. That means when we start the session it is focused on the research questions, not the problem area. We allow five minutes, at the start, to agree on what the team are working on. We also check if everyone is familiar with Miro, getting them to pick and write on a post-it.
Preparing for research
Miro has changed the way we prepare for research sessions. We’ve been trialling a new process for usability testing recently that has created more upfront prep for the team where they use their hypothesis to create the test scenarios. We have found it helps to have all this on one infinite whiteboard along with screens of the prototype being tested, so when research happens it’s easy for the squad to follow what is being tested and what to look out for.
Running research
We’ve experimented with a variety of remote testing tools. I won’t write a full review of each of them as a user researcher will be much better placed to do this, but I’ll explain how each was useful to us.
We have used a range of moderated tools including Hello Ping Pong, Lookback and Usertesting they all have pros and cons.
Hello Ping Pong, is great for straightforward usability tests though the participant pool is small, recruitment is quick, and the interface is intuitive. It has limitations for any complicated screeners, or for testing prototypes on mobile (it’s possible but laborious for the participant and researcher but the company are working to fix this).
Lookback has the advantage that it’s a one-off yearly payment. It allows us to moderate, stream and record sessions. But we need to use a separate company for participant recruitment so we use it for more complicated screeners.
We’re only starting to trial Usertesting.com, but we expect it to meet most of our needs. The recruitment pool is reportedly the largest in the world and, based on a first few initial tests, it’s been impressive.
The greatest advantage of remote research is that it helps us to extend our catchment area beyond London, and by reaching into people’s homes with little disruption to them we’ve noticed greater diversity in our pools. Parents, older demographics, people with access needs that may have never been able to travel to a lab in central London are now more easily included.
When we do return to the office, we won’t abandon our physical lab completely. In fact, we are building two in BT’s new offices in London and Birmingham (more about this in a further post). Being face-to-face allows researchers to observe more behavioural cues, build a relationship with the participant, and until everyone has great internet, technology still creates some barriers like connections suddenly dropping.
Analysis
After each session, we facilitate collaborative analysis. Again, this is also quicker and easier because of the prep the squad has already done to get the Miro board ready. Of course, we do still create and playback a report but by this stage, the team has been fully integrated into what we are trying to learn and the outcomes.
So, what are our lockdown lessons?
Good internet connection for the researcher is important (I can recommend a great internet service provider!). It still won’t help if your participant has poor internet but at least you won’t be the one dropping out. We’ve noticed testing prototypes from Figma uses more data than testing a live site, so you might need to allow a bit of time for this. Though we have had mixed experiences across tools.
Some of the team have said that a second monitor is important so they can keep an eye on the team chat / Miro board whilst facilitating a session. Basically, make sure your team have got the kit they need to function effectively from home.
It’s also important the team get to see friendly faces on a regular basis. User researchers don’t always attend product squad stand-ups as we work in a federated model, so we introduced our own daily stand-ups during lockdown. Initially, they were every day at 10 am but we reduced them to three days a week, and that seems to be sufficient to check-in without slowing things down.
We health check once a month with a retrospective so we can tweak what isn’t working. We also have a weekly team meeting, alternating this time between running deep-dives (focusing on a method or problem) and reviewing and planning our capability objectives to build skills and understanding across Consumer Digital.
Thinking back to how my team have adapted to working in a completely different way through a crisis makes me feel really proud. It’s been hard and we’ve experienced difficulties, but we have learnt lots and I know it has helped us to grow.
What have been your big lessons from conducting research remotely? Comment and let us know.
