I remember the conversation that sparked it all, the one that set the course for the most high-impact work of my career. It was the day in 2012 that a senior member of our research team asked, “How many customers do we have on our research panel?”
I blinked and waited for my brain to formulate a response. The truth: I had no idea, and it was complicated. Yes, we had access to an endless list of customers for Salesforce’s CRM software, but no clue how many might participate. In that pre-GDPR world, any such answer was always a shot in the dark. Yet our need for user insights was growing as fast as our team’s headcount, and we needed infrastructure to support both.
While I didn’t recognize it at the time, that day was a turning point and my teammate’s question fuel for the fire. We had to find a new way to manage participant recruitment.
The Early Days of the Field
But back then, research operations wasn’t yet an established practice. Publications and reference material were scarce, and there were few others in similar roles. So we looked inside our organization for inspiration, seeking to understand the needs of Salesforce researchers and UX program managers (our research recruiters), as well as the available technology.
We were at Salesforce, and there was one clear option: Build on Salesforce.
The Story of a User Research Platform
Before I describe our platform, let me note three things:
- Our team is not technical; we partnered with an external development team to implement our features.
- The features we’ve developed are not proprietary; they can be customized using existing Salesforce technology with additional development.
- Our database evolved in response to our internal process flows — a process that’s taken multiple years, iterations, and financial investments to get where it is today.
And now, I’ll share some favorite features built to support our research operations workflows:
Research Sign-up Landing Page
Our first challenge was to learn more about our participants and how they were using Salesforce products, so we could match them to our research initiatives. Though we were using existing survey tools to administer screening questions, we still needed a way to collect a wide range of information upfront, to avoid asking repetitive questions over time.
So we built a sweeping survey covering all core Salesforce products, with the goal of finding out who used which products — and to avoid missteps like inviting sales users to studies intended for developers. We launched the research sign-up site using Salesforce Sites, and continue to use it today. Responses from this page help us curate a refined list of potential participants, whom we then invited to participate in suitable research. We’ve since adapted the signup flow to accommodate shorter questionnaires, for when transaction time is limited.
Managing Participant Consent
Managing consent is a key part of maintaining global privacy compliance. Our participant database complies with global industry standards and data protection regulations such as GDPR and CCPA. Through it, participants can consent to be contacted, and acknowledge program terms and conditions. They can also unsubscribe at any time, via the site or from any of our communications.
Recruiting with Personas
Across Salesforce, we often use personas to describe specific groups of users. Our research operations team uses them to help determine who might be a good fit for our studies. To support recruitment, we added custom database fields to assign user personas, based on responses within the sign-up form. The system assigns a persona to each respondent, saving our recruiting team lots of time.
Distributing Incentives
Incentives management can be a headache — but it doesn’t have to be. We distribute honoraria to participants all over the world through our database, via a Custom Object that emails gift cards to users with just a few clicks. The database tracks who’s received what, and we use the year-end reporting features to identify which participants, if any, receive more than $600 in any given year and need tax-reporting information.
Enabling Self-scheduling
Imagine sorting the availability of 60 study participants and keeping track of the associated input. Scheduling research sessions can absorb administrative overhead and quickly become unmanageable. Our solution was a self-scheduling portal: Each participant receives a unique link from our Salesforce instance and is instructed to select a session date and time. We manage time slots from the Salesforce instance, and can quickly update them to match researcher availability; participants see updates in real-time. Shifting scheduling ownership to participants lets us operate more efficiently and scale with our growing team.
Tracking Participation Metrics
Metrics have such power. They bring glory (or shame) to any business team, and research operations are no different — maybe more so, because the researchers we support really love data. The metrics we track help us understand the health and diversity of our database. We know how many studies we’ve conducted, how much we’ve spent on incentives, the percentage breakdown of personas on any given panel, our busiest research season, our recruitment throughput, and much more. This data helps us forecast trends and position ourselves to react to research requests over time.
Maintaining Panel Health with Automation
If you manage a system, you need to keep data cleanliness in mind — especially with self-reported data, which can quickly become irrelevant or outdated with time. We first responded to this challenge by sending an annual solicitation asking members to update their profiles. But as the panel grew, this solution didn’t scale. So we adopted a concept called Automatic Profile Refresh — the database sends a refreshing reminder on the anniversary of each member’s signup date. No response triggers another email at the two-year mark; if that one’s ignored, the participant is unsubscribed. This not only encourages updates in a programmatic and scalable way, but it also improves database health by removing inactive members and helps us manage the consent lifespan per our data privacy initiatives.
Growing Through Referrals
Our panelists are surrounded by other Salesforce users, each with valuable opinions about our products. Our referral program rewards participants for referring colleagues, friends, or family members, sending each a unique referral link generated from our database. Participant communications also encourage them to spread the word. The result is a panel that has grown exponentially.
Share your Stories
The options for customization in Salesforce are almost limitless. We’ve used them to grow a successful research operations practice while accelerating user research here at Salesforce. Whether you’re launching a new participant database or evolving a mature solution, we hope you’ll share your stories and progress with us!
Want to learn more about Salesforce UX? Read our other Medium posts, follow us on Twitter @SalesforceUX, or contact uxcareers@salesforce.com for job opportunities. Then check out a post from my colleague Alyssa Vincent-Hill on developing data-driven personas at Salesforce.
To share your Salesforce feedback, sign up for our user research program.
Special thanks to Nalini Kotamraju and Jason Kriese for creating the space for ResearchOps to succeed, and for their endless encouragement — and to Aviva Rosenstein for asking the right question.
