Increasing ResearchOps Impact Through Automation
How our small research operations improved efficiency and increased org-wide awareness by automating key steps in our research process.
Research Operations (or ResearchOps) is often considered a tactical role/practice, sometimes requiring a lot of back-and-forth between parties and eating up chunks of time. This not only can be tedious for your Ops team but becomes increasingly troublesome when the demand for research in your company continues to grow despite your team’s limited capacity.
Here are some of the key tasks that our ResearchOps practice at Auth0 decided to automate in order to free up some of our bandwidth, as well as how we leveraged existing tools to do it.
Identifying the Problem
In our day-to-day work, we spent a significant amount of time on fairly repetitive tasks: performing check-ins, running after stakeholders for updates, and conducting post-project retrospectives with researchers. For a lot of teams, this is good enough for ResearchOps: tactical support and enablement. Instead, we view ResearchOps as a strategic practice, so being stuck on manual work severely limits our impact.
This tactical workload also kept us from being more timely and proactive during the early stages of the research process, especially when it came to responding to research requests and recruiting study participants (which has downstream effects on research velocity). Beyond that, it kept us from thinking more strategically: how to break down silos, shorten the feedback loop, and share knowledge across product teams.
Uncovering Opportunities
Once we discovered how much of an impact this was having on our Operations team, we set out to identify automation opportunities that could be categorized under three main buckets:
- Keeping track of Research Projects
- Ensuring everyone knows their current project status
- Sharing and activation of research findings
In order to identify the necessary triggers, actions, conditions, and desired outcomes, we mapped different workflows for each of the above categories. We then implemented these using Workato, an automation platform that is thoroughly documented and integrates nicely with our existing toolset (notably Jira and Slack). Because Slack is Auth0’s main communication platform, we decided to drive most of the interactions through it. Slack also provides an abundance of integration and extension capabilities.
After a few iterations, we wound up with four workflows. Here’s what life looked like before and after each:
Processing Research Requests
Before automation
Most often than not, one-off notifications from Jira were being ignored and forgotten depending on our workload priorities. This resulted in bottlenecks and delays when responding to research requests.
After automation
When a request for a new research project is submitted, both the Reviewer (usually our Research Strategist) and the Requester are greeted by a chatbot confirming the form submission. Going forward, whoever the project is currently waiting on will get another DM from the bot every 48 hours. This ensures that requests are reviewed in a timely manner and helps avoid bottlenecks in our pipeline.
Supporting Participant Recruitment
Before automation
Auth0 is an Enterprise SaaS company, meaning our Customer Success team has a strong say in how we engage with those sometimes-multi-million-dollar accounts. This, coupled with the CS team sometimes lacking enough context or having competing priorities (after all, their job is to keep customers happy, not help us recruit), would often delay our projects for multiple weeks.
After automation
As soon as a request is submitted, the ResearchOps team is notified so they can review the request and begin to prepare a Recruiting Plan. The Insights Request also contains an embedded recruitment criteria form that we can directly map to our customer database, increasing recruiting accuracy. Alongside these process improvements, we were able to establish rules of engagement for which customers can be contacted directly without getting CS involved.
Sharing Research Findings
Before automation
A lack of standardization of how Research Findings were communicated across the organization contributed to the issue of siloed research. Different formats, different channels, different cadences; nobody knew where to look or what to look for.
After automation
Workato & Slack to the rescue once again! Researchers receive a form in Slack halfway through their study prompting them to share progress and updates. Once research is complete and added to our Insights Repository, Workbot compiles a summary and shares it in a few key channels in Slack.
Keeping everything in Slack reduces context switching and helps provide teams with timely updates. It also provides a lightweight way for everyone in the company to see what research is happening, learn more about our customers, and engage with a variety of our product teams.
Tracking Research Impact
Before automation
One of the drawbacks of running a heavily democratized research practice is you don’t always get to close the loop. The best way we had of capturing the results and impact of the variety of Research that happened at Auth0 was a bi-annual Research Engagement survey, leading to both sparse and infrequent updates.
After automation
A couple of weeks after finishing every project, Researchers are prompted to fill out a form with questions about the outcome of their research, what impact it had on their team, and what feedback they have for us on the process. Having a constant stream of updates has drastically improved not only our impact tracking but also our overall Research process.
Reaping the Benefits
In as little as weeks after implementing these automations, we began to notice improvements:
- First, the Operations team, now freed from (some of) the burden of tactical support work, leveraged their new bandwidth to address research requests in a more timely manner, significantly decreasing the average first response time for request tickets in Jira.
- The automated reporting also increased awareness of research findings across different teams, generating some buzz in Slack with each bot post.
- At our next Research Engagement survey, we saw an increase in the perceived value of automated workflows as well as an increase overall in research engagement and satisfaction.
Great! Let’s automate everything!
Not so fast. The one thing automation will never replace is human interaction. While we’ve seen a lot of benefits, we’ve also noticed some drawbacks. People will tend to deprioritize messages coming from a chatbot in Slack, resulting in lower engagement, particularly for notifications taking place after research has been wrapped up (such as retros or post-mortems). This increases friction in tracking research impact across the organization.
We’ve also realized quickly that, as soon as our processes change or evolve, this creates new opportunities for automation. Your process should evolve to meet the changing needs of your People Who Do Research (PWDR).
What’s next
With all of this in mind, we are currently planning our next batch of automated workflows. These will reflect recent changes in our recruitment process as well as the implementation of research KPIs defined to track the velocity of each step in our research process. Increased visibility will help us home in on the opportunities which have the greatest impact.
