Conducting Effective UX Research

In our research, we strive for the right combination of rigor, structure, and flexibility. We love scripts, but we’re not married to them. We define roles for researchers, but we don’t always stay in our lanes. We meticulously document every bit of collected data, but prefer to share insights conversationally.

We’re still relatively fresh in these processes and are still establishing our norms. Here are a few things we’ve learned.

Note: Keep in mind that Sprout Social is B2B, which comes with its own set of challenges and nuances. When necessary, we’ll note where being B2B affects our approach.

The more the merrier

Effectively facilitating, listening, and note-taking a call by yourself is a tough task. If you focus on one aspect, the others suffer — which is why we’ve found that it’s useful to have at least three people in the room. Each person can focus on a particular responsibility during the call. Keep in mind, though, that these roles can shuffle around on-the-fly to accommodate changes in the flow of conversation.

Facilitator

The facilitator provides the thread that runs through the entire conversation. They introduce the call agenda, they keep the call progressing, they ask questions from the script, they transition into and out of prototypes — they drive the conversation.

Listener

The listener’s primary responsibility is to listen hard. They note interesting or unexpected feedback and help the facilitator unpack it. They’ll often hop into the conversation with a “why” or a “how” question in an attempt to uncover a deeper insight. This role’s shape often varies the most from call to call.

Note-taker

The note-taker is there for detailed note-taking with some real-time synthesizing. A note-taker will often have a timer synced up with the call recording to keep track of key moments in the call notes. They often remain mostly silent during the call — afterward, though, they help lead the post-call synthesis.

Note: You may not have the resources available to dedicate three team members to each call. Not to fret — there are approaches for effective two-person and solo calls as well. For a two-person team, you can have a facilitator and the other person can be dedicated to listening and note-taking. If you’re by yourself, record the call and focus on being a present, engaged facilitator — you can re-listen to your recording to take more thorough notes, if necessary.

In our calls it’s been common to have a Product Manager as a facilitator and Product Designers as listeners and note-takers — but it doesn’t have to be this way! With a well written script and clearly defined goals, any member of the team can take on any role. In fact, over the course of a call, we’ve found these roles often change on-the-fly. This is especially the case during prototype walkthroughs when a designer will switch from note-taking or listening to facilitating the walkthrough, where then the call’s main facilitator takes notes in return.

Having clear expectations of everyone’s role during the call helps provide focus. When your responsibilities are manageable, you’re able to focus on the most important part of the call: the customer.

Find the right customers to talk to

Finding customers is a bit like matching supply & demand. There are people (customers and users) that you may want to talk with, based on your demand or needs, and there are people who are available to talk, based on the supply we have access to.

Demand: Who do you need to talk with?

Are you looking to talk with a particular type of customer? It’s helpful to consider the relevant qualities of your ideal candidate.

• Industry: Airline, hospitality, retail, government, university, etc.
• Location: Your location or anywhere?
• Segment: Agencies, Small-to-medium sized businesses, Corporations, etc.
• Plan types: Does your product have different plan levels? (Standard/Premium/Corporate/Enterprise/Agency ) Consider whether your feature will eventually have plan assignment. It may not be bad to talk with someone on a different plan, but take that into consideration and communicate it to the relationship manager, so that you aren’t setting up customers to provide valuable insights and receive no benefit in return.

Are you looking to talk with a particular type of user?

• Should they be active users of a particular feature of your product? This can be helpful so that you can get relevant feedback from a user who is exposed to and familiar with the feature.
• Do you want users who would get value from a certain feature but don’t use it? This can be helpful so you can better understand their hangups.
• Consider that sometimes managers have a different role than their subordinates, and thus may have different usage.

For example—on our squad, we were conducting research for a feature that would help alert users to abnormal spikes in activity. We were looking to talk with:

• Customers who had a high volume of incoming messages
• Users who actively used Sprout’s Smart Inbox (rather than managers or people who primarily publish — though that can also be valuable)

Supply: Who’s available and ready to talk?

Once you have a clearer picture of who you want to talk to, you can start recruiting with purpose. You’ll want to consider possible participants based on a multitude of factors.

• User feedback
• Users who have proactively asked for a given feature or expressed a particular problem
• Quantitative data sources
• High volume usage: users who have performed certain actions
• Low volume usage: users who don’t use some feature regularly — in case the goal is to understand why a certain feature isn’t used or why users abandon

You can explain the problem area and general feature request, and solicit suggestions from your own customer success team on who to talk with, because they often have a good pulse on their customers’ use cases and common problems

Making a match: Demand meets Supply

Basically, the entire matching process comes down to this:

1. Make a list of customers and their Relationship Managers (RMs)
2. Reach out to the RMs of those customers
3. Coordinate calls with the RMs and their customers
4. Reserve a space for your call (don’t forget this step!)

Learn as much as you can beforehand

Knowing who you’re talking to is just as important as how you’re engaging with them. Taking the time to understand the bigger picture before the call gives you a leg-up in understanding their context. Pre-call intel can include looking through information in the following resources:

Your company’s CRM

• Use this to see notable activity and recency of the customer to see whether they’re new to Sprout and whether they may have been using other competitor tools before

Quantitative Data

• Use this to see interesting product/feature usage, so that you know a bit more about how they’re using the product and can ask intelligent questions on why they might be doing something

Direct User Feedback

• Use this to see what they’ve previously submitted about the specific feature your’e discussing or if they’re inquired about related features. For customers, this is an opportunity to talk to the product team, so they may or may not care what your particular focus area is.

Google

• I’m sure you know how to do this, but it’s a good reminder.

Be prepared for the call

Once you’ve found the right customer to talk to, learned about their usage and history, and done a basic review of their brand you’re nearly ready. You’ve just acquired a boat-load of information — now is your chance to use it by crafting a script for your call.

Stay on script

Having a script for your call provides a few benefits: it provides consistency across calls, helps your team align on goals, and serves as a life-preserver when things go awry.

We’ve found it useful to highlight call goals at the beginning of script, followed by a welcome (to segue from introductions), with sections of time-estimated questions after.

Can you hear me?

Always call into a customer call. Everyone may be on the same call, but be sure to get audio in/out via a dedicated phone. This may seem small — but when this goes wrong it’s a call killer. Imagine this nightmare scenario: everyone leaning into the same laptop, barely hearing the customer, feeling uncomfortable and frustrated — not a great headspace for a productive call!

Fill in the imaginary blanks

Including logos and brand names in wireframes and prototypes can provide some familiarity for users to help them give the right feedback. We’ve all heard of stakeholders getting hung up on Lorem Ipsum text, it’s the same idea — use real content as much as possible.

Take great notes

The note-taker should have the time and space to take detailed, concise notes.

That’s it. You’re ready for the call. Follow your script. Listen hard. Empathize with this particular user — this particular human! If you’re present and taking great notes you’ll have a wealth of information to distill insights from.

Synthesize insights (soon) after the call

Synthesis is what we call the stage where we digest our notes from customer calls to distill insights and takeaways. We do this for two audiences:

1. Ourselves, so that when we think back to the call or overall research, we can recall the high-level takeaway, and
2. For people that weren’t directly involved but would benefit from knowing the takeaways.

We’ve found it valuable to do two types of synthesis: post-call synthesis and overall synthesis.

Post-call synthesis is an exercise to digest an individual call’s insights. We’ve typically done this exercise for 20 mins, preferably right after the call is complete — while insights are still fresh and we still remember the context of the insights. It should be relatively short — we don’t need to review the entire call. Collect the insights which are most important for the team to take away.

Where possible, we identify the highlights that will actually influence a decision we’re trying to make. Sometimes the highlights are broader and could be about high-level concepts that we are also thinking through.

We like to document these with interview notes, so that the insights are easily accessible, and full details are below if someone wants to dig in deeper.

Overall synthesis is an exercise to digest the aggregate trends across the calls and notable takeaways that can be applied broadly.

Share your insights

Truthfully, this part of the process — the most important part, is what we have the least experience with. Sharing insights is how the knowledge spreads. It’s how you establish a culture of research. We’re employing a two-pronged approach: Documenting all research in a single Research Hub, and sharing insights more atmospherically.

Research Hub

In practice, notes are taken via a variety of methods and mediums. Having a place to collect them all, and show how insights were gained is valuable. A research hub provides that structure. The key to this part of the approach is that it’s accessible, searchable, findable, and easily digestible by as many members of your team as possible. Your hub could reside anywhere and in any form — it could be a shared Google Drive, a shared Dropbox folder, a pinned set of messages in Slack, A Confluence space, a wiki, etc.

Atmospheric Sharing

In reality, your teammates will rarely seek out research insights on their own. It’s your responsibility to share what you’ve learned. Sometimes it’s as simple as sharing a single insight from a recent call on Slack or sharing a quote from a user during standup. Sprinkle insights throughout your team’s collective spaces; over time, your team’s foundational knowledge of and empathy for users will grow — and ideally, it will help your team be more certain in their decisions.

Bonus: When things go wrong

Example scenario: You veer off track into interesting discussions and run out of time

Being aligned on the goals of the research call and the overall research study/initiative can help to gauge, in the moment, whether this is truly a problem or not. There may be situations where it’s okay to veer into higher-level / meta discussions that are off script, but a) try not to keep the customer for longer than they had scheduled to be on the call, and b) try to evaluate if that puts the actual research study at risk. Some options/guidance:

• If there are other customers you’ll be talking with, from whom you can still extract necessary insight, then perhaps it’s okay for this one call to veer slightly off the beaten path.
• If you have some urgency around the insight you need, try to tie up the thread of the conversation and get back on track before you run out of time.
• If you end up running out of time, then your best bet is to see if the customer has more time on that call, or to find another customer to talk with. Given that we have a pretty extensive customer base, this may not be a daunting task.

Ultimately, you control the timing and pace of the call, so you need to own this decision and serve the best interests of the research study and the customer relationship.

Example scenario: It turns out that the user’s use case may not fit with what you’re prototyping

This might be a great learning experience, or it might be a reason to end the call and save some time. This comes down to a judgment call, especially by the facilitator on the call and the relationship manager. A few notes/tips:

• It depends a bit on how far off the user is from the use case.
• If the user is the wrong role (e.g., they’re a manager, but you wanted a team member) or they have no insight into how a particular feature works, and if your hope was not necessarily exploratory testing with “fresh eyes”, then it might be a good idea to see if you can work with the RM to find out who might be the right person for the customer and schedule a different call.
• If the user uses the feature/feature area, but they use it differently than you anticipated, this might still be a good learning experience to discover a new use case and/or see how our decisions affect other parts of the app that we didn’t anticipate.

Example scenario: The user seems really quiet or is having trouble answering questions

This is where the momentum of the facilitator and the listener really help to double-down on extracting insights from the call. Without improvising a little bit and digging, you may end up getting through all of your scripted questions, and still not have a comprehensive understanding of the user and why they do the things they do. Some tips:

• Try to conversationally dig for additional why’s and how’s.
• Although the idea is not to answer questions for the user, try to give them scenarios and example options to react to.

Example scenario: There are multiple users on the call, potentially across different teams and at different hierarchies

• Try to learn who the distinct users are and what roles they have (at least at a high-level)
• As you ask questions throughout the call, try to see if you can direct the question at a specific person, based on who you think would most likely answer the question — just so that they don’t have to waste time with uncertainty. If you’re wrong about who should answer the question, that person will probably say, “I’m not sure, Suzy might have better insight here”.
• With more people involved, you have to manage flow a little bit more. Customers will step up to the plate and engage, if the flow feels productive.

Bonus Bonus: TL;DR

Assemble a research group. If you’re by yourself, that’s OK, too.

Find the right customers to talk to.

Learn as much about them as you can beforehand.

Prepare for the call: write a script, know your goals.

Listen hard, be present, ask questions, take great notes, record the call.

Synthesize insights from your notes right after the call.

Share your insights.

✨

This story was co-written with the help of Anjali Thakrar and Jimmy Yoon.