What Research Ops Professionals Are Doing in Response to COVID-19
COVID-19 has changed our lives in significant ways: many of us have been asked to work from home and limit social contact; countless people’s financial lives are in the balance; we’re living in uncertain and anxious times.
For the lucky of us, work continues and, if you’re a researcher, that means research continues too. But unless you work in a digital-by-default organisation, how you do research — how you recruit and communicate with participants, gain informed consent, encourage observers and teammates to get involved, synthesis and analyse, share your research and more — must change too.
Research Ops professionals around the world are meeting the immediate challenge of research during COVID-19 and seeing the long-term opportunity to build practices that are remote-friendly, robust and resilient to change.
In this post, thirteen Research Ops professionals share how they’re adjusting their operations to suit the times.
Helping researchers make sensitive decisions during COVID-19
While Research Operations often focuses on delivering infrastructure and support to researchers, we also work closely with researchers and other stakeholders to make sure practices are ethical and empathetic, something that’s important now more than ever.
Stephanie Marsh, UX Research Operations Manager at Springer Nature:
We’re helping teams decide whether now is an appropriate time to be doing a specific piece of research. We’re having conversations about being sensitive to participants’ contexts and mindful of the potential of data being skewed by current themes.
Springer Nature’s main user groups are scientists, students and librarians. In the beginning, when labs and physical libraries are shutting down, people will be very busy organising how they’ll work in the coming weeks and months. They may start experiencing issues we previously haven’t anticipated and different from the problem spaces we’re currently exploring in our research.
To help with future recruitment strategies, we’ve started tracking participant cancellations to monitor how drop out rates fluctuate in the current climate. We’re also helping researchers identify whether there’s been an impact on participant availability as the situation progresses and changes over time. This will help us all plan better in terms of how long a piece of research might take. In some cases, we need to extend timelines.
Stig Parfrey, UX Research Programme Manager at Workday, is tracking responses from representatives of different customers to understand how specific companies are affected and if it’s appropriate to reach out to them with research participation requests.
Steven Hill, Research Operations Manager at Booking.com, works with participants in the hospitality and travel industry, which has been significantly impacted:
In addition to the logistical and operational considerations that go into planning (or re-planning) a remote study, researchers are looking to Operations for guidance on when it’s appropriate to continue reaching out to users (both consumers and businesses) and when it would be best to pause and realign.
It‘s become an important conversation between Research and Research Operations to coordinate on a project-by-project basis and encourage researchers to consider current priorities, recruitment needs and participant bias.
Maddy Vasquez, User Research Coordinator at Deliveroo:
“My city is under quarantine, should I keep working?” or “The customer will not take the food, they suspect I’m ill. What should I do?”.
We’ve teamed up with our Communications team for riders and restaurants to create a quick FAQ page for researchers to refer to during research sessions. As the situation is evolving rapidly, we want to make sure that researchers are confident they’re able to provide accurate answers if common questions come up during a session. This has eased some anxiety for our researchers.
Annie York, UX Research Ops at Kabbage:
Our customers are small and midsize businesses (SMBs). Unfortunately, SMBs have been some of the hardest-hit populations since the outbreak of COVID-19. As we don’t usually recruit outside of our customer base, we want to be sensitive to the amount of contact they’re getting from us as a company. This has caused us to scale back on the amount of UX Research we’re doing.
We recently piloted a program called ‘Research Fridays’, in which any team member can chat with customers about the topic of their choice via remote video chats. Given the circumstances, we had to decide whether or not to continue the program. We saw two sides of the coin: on the one side, we felt incredible empathy for how overwhelmed our SMBs might be feeling and didn’t want to contribute to the number of emails piling up in their inbox. On the other hand, we felt we could perhaps help by softening the financial blow via participation incentives.
After much thought, we decided to temporarily halt the pilot. We always want to be a source of understanding and partnership, not stress, for our SMBs.
Recruiting research participants during COVID-19
It goes without saying that our research participants’ lives have changed radically too. They may have more (or less) time on their hands, there may be people and noises in the background, they may need to cancel last minute, and so on. Researchers and recruiters are taking this into account and adjusting their strategies and messaging to both help research go smoothly and put potential participants at ease.
Noel Kirmse, Senior Manager Research Operations at Salesforce:
We’re finding that we need to be flexible with recruitment — in accessing participants, recruitment criteria, rescheduling sessions, extending study timelines, accepting that there may be no-shows etc. and being compassionate and understanding of each participant’s situation.
We’re now including this message in our research recruitment communications giving people the opportunity to quiet their inbox from research invites:
Feedback from our participants has always been at the heart of everything we do. We understand the challenges some are encountering as COVID-19 surfaces around the world, and realize this may not be the right time for you to participate in one of our research studies. If you’d like to be removed from our invite list, even temporarily, please let us know. The health and safety of our customers will always be our most important consideration, and we commit to supporting your needs to the best of our ability.
Molly Fargotstein-Sanders, Associate Research Operations Manager at Mailchimp:
We’ve taken a very thoughtful, careful approach to handling recruitment. While we want to stay on track and get in front of the customer, we understand that this is an incredibly difficult time for so many, if not all, of our customers. Much like Annie at Kabbage, a large number of people who run small-to-medium businesses use our services, and we want to be super sensitive to that.
We’ve begun a more personal recruitment approach, honing in on smaller groups of ideal participants versus casting a wide net. We want to be intentional about reaching out to the right people in a thoughtful way, so, if it takes a little more screening/criteria confirming on our end, that’s okay with us.
We’ve shifted the language in our recruitment efforts to clearly articulate how voluntary this opportunity is, highlighting the priority of health and safety over their participation.
We don’t want to gloss over the situation we’re all in, because we want our research participants (or potential participants) to feel supported and connected to Mailchimp, however they want to be. At the end of our recruitment email, we provided links to helpful resources on our website that cater to COVID-19 relief. Even if they decide not to participate, it’s our duty to help them out.
We always extend the offer to pause recruitment communication with them until a more ideal time, so they know they aren’t closing a door if they don’t want to. Because, of course, research participation opportunities will still exist on the other side of this.
Stig Parfrey from Workday shares:
At Workday, we’ve drafted templated language for our researchers to use when recruiting and communicating with research participants to signal that we understand it isn’t ‘business-as-usual’ and to reassure everyone of our flexibility. We’ve made sure that the language is consistent with Workday customer communications on the issue.
Sarit Geertjes, Research Recruitment Coordinator for Atlassian, has created templated communications and guidance for all Atlassian’s to use when reaching out to potential participants:
Atlassian is now conducting all research sessions remotely; the health and safety of our research participants is of primary importance to us. As COVID-19 surfaces around the world, I understand that your challenges, contexts and needs may be changing daily. Please let me know if you need to cancel or reschedule your session and I’ll do my best to find a more suitable time. As schools and workplaces may have closed, I understand if there are other members of your household in the background during the session.
Fatimah Richmond from LinkedIn:
With the world around us changing so quickly, we’re considering creative ideas and strategies to gather input from members and customers to quickly inform how LinkedIn can better address the impact of COVID-19.
LinkedIn is a members-first company, so our recruiters and schedulers align across the company so that outreach communications are mindful of the participant experience during this time of uncertainty.
Thank you gifts (incentives) for remote research
Kate Towsey, Research Operations Manager at Atlassian:
My team use Tremendous to send e-gift cards to say thank you to participants. Many research teams use Tango Card, but we’ve found that Tremendous is quicker to admin for studies done internationally.
For participants who can’t accept a monetary thank you or prefer an alternative, we use SwagUp to produce and ship Atlassian swag boxes. Though we’ve got that on hold right now to alleviate strain on postal services.
(At all times, and for legal reasons, we don’t talk about ‘paying’ or ‘incetivising’ participants, instead, we ‘thank’ them for their time.)
Making informed consent remote-friendly
As our research sessions are remote, so our informed consent processes must be remote-friendly too.
Theresa Marwah, Research Operations Technology Lead at Atlassian:
Our brand new consent process is 100% digital (and just in time!). The informed consent guidelines we’re currently rolling out to researchers is designed for remote research scenarios, as this is the bulk of our practice.
We use Qualtrics to manage digital consent — both the signing and storing. Once the consent form is signed, Qualtrics generates an anonymous participant ID that we tag to research materials so we can trace the consent against the data we collected, such as a video file. This traceability enables us to protect personal data and uphold our obligations for right to be forgotten (RTBF) or deletion requests. Using Qualtrics enables us to manage permissions, generate reports and automate look-up and delete requests via APIs from our central privacy systems.
Springer Nature uses DocuSign for digital consent. Stephanie Marsh shares:
Participants can read our consent form and add a digital signature, it works really well. We ask participants to sign the consent form before their session and researchers will ask at the beginning of a session if the participant has any questions about the consent. There’s different functionality you can use for digital pop-up research, too. For international research, we’re currently able to deliver the consent form in three languages.
Guppy Ahluwalia, Research Operations Manager at Dropbox recommends HelloSign as an alternative, it offers similar functionality to DocuSign.
Annie York from Kabbage:
We’re also 100% digital via Docusign. I always send the NDA prior to the call. In addition, our PWDRs (People Who Do Research, cred: Kate Towsey!) always ask for additional consent prior to starting the interview. We then store all of our consent forms in a shared Drive folder, that everyone, including Legal, can access.
Tools to support remote research
These days, many global companies are remote-friendly. Atlassian’s a good example: almost all of our research is conducted remotely, so our Research Operations are designed digital-by-default. We’re lucky. But many companies are needing to quickly tool up for remote-only workflows.
Kim Porter, Research Operations at Monzo Bank, a fin-tech company based in the UK, is working to enable completely remote research sessions for their research team:
We’ve added two extra Lookback licences to our toolkit to make sure there are enough licenses for the research we have lined up. Lookback allows for fully remote research, both moderated and unmoderated, with a facilitator and observers.
And we have a contingency plan in place too. Should more than three squads want to run research on Lookback, our backup plan is Google Hangouts. We’ve also written some emergency user guides and how-tos to help coach researchers in using these tools.
Theresa Marwah from Atlassian:
Atlassian uses Zoom for our team meetings globally. We also depend upon Zoom to host our remote moderated research sessions. Zoom offers a configuration that removes the need for participants to download software, it provides for waiting rooms, so you can control when participants ‘enter the room’ and we can make sure that observers are muted and their names anonymized before the session starts.
What we love is that Zoom now offers an integration with our video asset management platform, Medallia LivingLens. Our researchers can record a research session, and the video will be automatically uploaded to LivingLens ready for the researcher to generate showreels for sharing. This workflow also means that the personally identifiable data held in raw recordings are captured, transferred and stored securely. We’ve only just got LivingLens in place and we’re working with them to iron out some kinks in the workflow, but it’s given us a single and secure home for our recorded research data and has made the painstaking process of making showreels easier for our researchers already.
Fatimah Richmond, Research Operations Manager at LinkedIn, works with a team of Research Program Managers (RPMs), Schedulers and Lab Operations Managers (Lab Ops).
As a result of COVID-19, the Research Operations team is being asked to surface tools and functionality to facilitate the increase in remote research studies. Lab Ops is being asked to highlight how tools like BlueJeans integrate with streaming tools like Panopto. We’re also testing new remote video tools like Zoom.
Zoom includes delightful features, like funny backgrounds and reactions, this is important because humor is a cultural value at LinkedIn and can help improve morale during this stressful time.
Our RPMs are being asked to help with rapid re-prioritization efforts, too. They’re creating new ‘views’ in tools like Airtable to quickly inform User Research leaders as to what’s going ahead and what isn’t.
Remote research analysis
Operations teams are enabling remote team analysis sessions with conferencing and digital whiteboards tools like Trello, Mural and Miro in place of physical sticky notes, whiteboard pens and a wall.
Guides and playbooks are now even more valuable
Annie York from Kabbage shares why their Research Operations playbook has become even more important at this time:
Going 100% remote highlights the importance of keeping and maintaining a playbook for all of our PWDRs. We try and encourage our PWDRs to self serve as much as possible. This is achievable by creating and maintaining detailed step-by-step processes and how-to guides that cover anything from how to send a consent form to best practices when speaking with customers. Not only is the playbook helpful to those who conduct the research, it’s also tremendously helpful to our partners in Legal — the playbook serves as a reliable and immediate source where Legal could, for example, find all of our consent forms or our policies for recording and data storage. This creates a sense of accountability and ease for everyone.
The thing I am ‘noshing’ on now is how to make our playbook work for everyone. I’ve learned quickly that everyone absorbs information differently. I’m an auditory learner. Finding alternative ways to present the information will not only make the playbook a more useful resource, but will create less confusion during this time of separation.
Fatimah Richmond from LinkedIn:
While toolkits have always been a staple of the LinkedIn User Research team, our processes changed to support a team that became distributed overnight. Research Operations is being asked to structure content differently so that remote tools and vendor processes are surfaced and highlighted in online toolkits, like our Microsoft Sharepoint website. For example, we’ve highlighted our remote lab processes and remote tools like Miro.
Stig Parfrey, UX Research Programme Manager at Workday:
I’m sharing templates, tips and guidance for how researchers can best use the resources available for conducting research, workshops and other typically in-person activities. For instance, I’m creating a library of templates in the design tool Figma for journey mapping and research synthesis.
Supporting team morale and partnership
Even if the transition to 100% remote is technically easy, many people don’t thrive in socially disconnected contexts. As a result, companies are implementing initiatives to help staff members continue to be productive and feel supported while working remotely. But culture isn’t osmotic; it’s important that teams work together to create rituals that help keep them connected and motivated.
Vanessa Lo, Research Operations Coordinator at Atlassian, is using GroupGreeting to send team-signed e-cards to researchers as a way of celebrating special events. Due to COVID-19, our normal cake operations (CakeOps) has been shut down. 🎂
Noel Kirmse from Salesforce is supporting researchers with some novel activities:
My team is having regular virtual one-on-one check-ins with researchers to make sure they’re supported.
We’re also doing remote volunteering parties: teams are gathering virtually over Hangouts to complete remote-friendly volunteering activities together. A couple of examples include transcribing historical documents for the Smithsonian or mapping streets and buildings in remote Uganda.
We’re also playing Flat Stanley and embracing ‘snail mail’ — we’re sending cards or care packages to each other, specifically to team members who are celebrating special events.
Kim Porter, Monzo Bank:
We’ve scheduled daily ‘tea breaks’. These are informal (optional) 15-minute chats over remote conferencing, which researchers can join if they have free time and want a bit of social interaction.
We’re doing lots of sharing, too: we’re sharing morning routines within the team; things that make us feel calmer and happier; home-based workout plans; workspace tours; content recommendations for blog posts, shows and children's’ activities; daily picture challenges. We’re also going on walking 1:1s with Slack calls on our phones.
And finally, pondering silver linings
A question I’m sure many of us are pondering: What of our new remote lives will we want to keep when we go back to our workplaces?
In building remote- or digital-first operations, are we building operations that are more scalable and resilient to change? I think we are. If we’re smart, we’ll build our new remote operations so that they’re appropriate to in-person contexts, too, and our current (emergency) work will remain valuable in the long-term.
We may also find, more than ever, that we need to focus our operational resources on designing and deliver self-service pathways (recruitment, procurement, data management etc. platforms, vendors, templates, training and guidelines) for researchers, rather than seeking to do the work for researchers. Self-service pathways are extremely resource-efficient, they’re easier to deliver, manage and scale in distributed contexts, and they’re often empowering to researchers. There’s an argument to make what I call ‘highly-supported self-service’ a default operational strategy. It’s a strategy that‘s been very successful for me in FY19/20. And it’s especially relevant to our current times.
Written in collaboration with:
Noel Kirmse, Senior Manager Research Operations, Salesforce
Kim Porter, Research Operations, Monzo Bank
Stig Parfrey, UX Research Programme Manager, Workday
Kate Towsey, Research Operations Manager, Atlassian
Theresa Marwah, Research Operations Technology Lead, Atlassian
Sarit Geertjes, Research Recruitment Coordinator, Atlassian
Vanessa Lo, Research Operations Coordinator, Atlassian
Stephanie Marsh, UX Research Operations Manager, Springer Nature
Steven Hill, Research Operations Manager, Booking.com
Annie York, UX Research Ops, Kabbage
Guppy Ahluwalia, Research Operations Manager, Dropbox
Molly Fargotstein-Sanders, Associate Research Operations Manager, Mailchimp
Fatimah Richmond, Research Operations Manager, LinkedIn
