Running user research during a lockdown
On the 23rd of March, the UK Government made radical recommendations that every citizen should be staying at home, except for essential travel. This has immediate implications on our user research team — in both what our studies should explore, but also how we go about exploring them. In this blog post, I wanted to cover some of the challenges that lockdown creates, and how the user research team is adapting to meet them.
User research is still important
During times of uncertainty, both local and national news is very important. We’ve previously explored some of the motivations people have for reading news, and during the lockdown, current events have a clear and direct impact on people’s lives — informing people’s decisions about their personal safety, the safety of their family or helping to anticipate the impact that the lockdown will have on their livelihood.
This has led to an increase in many people’s consumption of news, and to support this our product and editorial teams are having to continue to make important decisions about Reach’s digital products, prioritising to use the finite time Reach staff have in the most effective way possible. Unlike many sectors, the demands on digital teams working in news are only increasing during these difficult times, so good decisions around priorities are more important than ever.
We strongly believe that having a better understanding of citizen’s context and evaluating how well our products meet their needs can help improve the quality of decisions being made by our product and editorial teams. Better information will reduce the time it takes to arrive at the best solutions for our users. This means that we’re still running studies during this time.
What impact does lockdown have on user research studies
Our first step when planning a study is to meet with colleagues from product and editorial and collaborating to define the objectives of each study — deciding what is the most useful topic to be learning about to inform their decisions. With the objectives defined, the method will then be chosen that best allows us to answer those objectives reliably.
Often the method we pick favours meeting participants in person. There are some benefits to in-person sessions. Speaking 1-on-1 with our users, and being able to pick up on non-verbal cues makes it easier to build a rapport with them, putting them at ease and potentially revealing more than we’d learn from a phone conversation. It also allows us to put a wider variety of things in front of them, including paper prototypes, making it possible to learn about people’s understanding and interpretation of our software before a single line of code has been written.
Our user research lab normally allows us to broadcast sessions hosted in our lab throughout the company, raising awareness and creating engagement which is harder to achieve when we’re no longer based in the office. Running studies at our offices also help us avoid a potential sampling bias — if we rely on screen sharing, we’ll only be able to speak to people who are digitally confident enough to use screen sharing software, which might not be representative of our readers.
Lockdown means that these in-person studies are no longer possible because they would require non-essential travel, and so we’ve had to seek alternatives.
How to run studies during lockdown
Like many companies running user research during these times, we have pivoted to remote methods — combining screen sharing via tools such as Google Hangouts or Zoom, with phone interviews to minimise the risk of excluding people who are not digitally confident with screen-sharing but still use our products. This allows us to continue running both usability studies — looking at whether people understand how something works and if they are able to successfully do it — and behavioural studies which look at people’s lives and how our products fit into them.
Being forced to run studies remotely does help us avoid another sampling bias we might normally be guilty of. Because we have a user research lab, there is a tendency to default to using it as a method. This makes it a lot easier for people who live in the areas surrounding London to participate in research, since travel to reach us is shorter. However, people who live near London’s behaviour might not be representative of people living in more rural areas of the country. This is a problem because we need to make sure our findings are representative of the whole country since Reach’s publications have a national footprint and represent many local papers. Although we do try to combine local and remote studies normally, this has reduced the temptation to default to London based studies.
What’s not possible currently?
We also need to be conscious of the context in which people are participating in research. Lockdown has changed people’s behaviour significantly, and is putting many people under additional stress as they worry about their family, their job, or have taken on additional care responsibilities. This means that their behaviour may not be typical — if people normally read entertainment news on their commute, they won’t be commuting currently, and it will be harder for us to learn about people’s ‘typical’ behaviour.
An impact of this is that we’ve had to be careful about our research objectives and make sure that we’re tackling sensible problems that we can apply when lockdown ends. We intend to achieve this by prioritising the studies where user’s behaviour won’t be as impacted by the lockdown and creating a backlog for the other studies to be tackled when the context is appropriate.
Looking forward
At the moment it’s unclear how long the current lockdown in the UK will last for, or when lab-based studies will be possible again. Even post the lock-down, it’s likely the economic impacts will lead to significant changes in people’s behaviours and opinions for some time after. During this time, our research team will continue to work with our editorial, product and design colleagues to identify the highest priority areas to explore and ensure that Reach understands the behaviour of its users, so it can continue to develop useful and usable products that meet their needs.
