Tips & Tricks for Successful User Interviews: A Guide for Product Managers

Learn the risks and pitfalls of insufficient methodology in user interviews. Learn how cognitive biases like overconfidence can skew UX research outcomes, and discover the importance of rigorous practices in gathering accurate user insights.

User interviews have become a cornerstone for deriving insights in product development and UX design. However, their effectiveness hinges on rigorous adherence to established methodologies. Without this, the risk of succumbing to cognitive biases increases, potentially skewing the data collected and leading to poor decision-making.

Understanding the Dunning-Kruger Effect in User Interviews

The Dunning-Kruger effect is a cognitive bias where people with limited knowledge or competency in a domain overestimate their own abilities. In the context of user interviews, this can lead interviewers to believe they understand user needs thoroughly without proper training or adherence to research methodologies. This overconfidence can lead to significant errors and oversight in gathering and interpreting user data.

Cognitive Biases You Might Already Have And Not Know

1. Confirmation Bias: This occurs when interviewers seek information that confirms their pre-existing beliefs. For example, if you assume users find an interface intuitive, you might ignore comments suggesting otherwise. This bias can be countered by asking open-ended questions that encourage diverse responses and by being receptive to unexpected answers.
2. Recency Bias: Giving undue importance to the latest information received can overshadow earlier data. If a particularly compelling opinion is shared at the end of a series of interviews, you might prioritize it over earlier, equally valid feedback. Overcoming this requires a systematic review of all data, not just the most recent.
3. Social Desirability Bias: Participants may alter their responses to align with what they think is expected of them. To reduce this, ensure anonymity where possible and create a non-judgmental interview environment that encourages honest feedback.
4. Anchor Bias: This happens when too much emphasis is placed on the first piece of information offered, which then colors all subsequent data. To avoid this, interviewers should consciously maintain neutrality and consider each piece of data independently.

Reevaluating Modern Feedback Integration Methods

In recent years, many companies have enthusiastically adopted new “methodologies “aimed at integrating ongoing user feedback into product development. However, this approach has often been applied without sufficient adherence to a robust methodology. They are more theory than practice.

These approaches have been touted as a revolutionary framework by product teams unfamiliar with the deep-rooted practices of user experience research (UXR). Unfortunately, this has led to a superficial application that lacks the rigor and depth necessary for genuine insight.

While conceptually sound with its focus on customer-centric research and cross-functional collaboration, often fails in execution. They skim UXR best practices, resembling a reinvention of established methods without a proper foundation.

This results in a diluted process where:

• Multiple team members crowd the interview sessions, disrupting the rapport-building essential for genuine feedback. Proper collaboration should occur at the beginning when setting goals, and later during the analysis and reporting phases — not in the midst of sessions.
• The frameworks suggest that anyone, regardless of their training in UXR, can conduct meaningful user research, which undermines the specialized skills required in this field. This notion trivializes the years of learning needed to master techniques such as framing, mirroring, forming research questions, hosting interviews, adapting to various situations, probing, and coding data.
• Companies might cut costs thinking they are conducting research correctly, yet they end up biasing participants, asking leading questions, not properly analyzing data, or skipping essential workshops for problem discovery. The depth needed to truly define user problems and needs is sacrificed for speed, leading to ill-founded product decisions.
• Teams employing these methods often skip vital steps such as creating empathy maps, defining and writing problem statements, or properly investigating How Might We (HMW) questions. They rush into identifying “opportunities” without investing the necessary time to truly define the problems, which is a critical yet challenging step because of the rush to move into solution development.

Ultimately, for companies that are extremely tech and product-driven, any methodology that encourages the acknowledgment of user research could be seen as beneficial. However, with improper execution, these companies risk obtaining incorrect data, creating the wrong products, and ultimately failing.

The ‘one-size-fits-all’ approach neglects the nuances of professional UXR, where significant training and experience are crucial for effectively framing questions, managing interviews, and analyzing data.

Oversimplifying UXR to make it appear accessible without proper training these modern methodologies undervalue and potentially sideline professional user researchers.

A counterargument might claim that such frameworks offer a quick way to gather insights, preventing teams from getting stuck in perpetual research phases. If that’s the case, it likely points to poor UXR practice or mismanagement rather than an inherent flaw in the research process itself.

Proper UXR is meant to inform and enhance product development, not to serve as a never-ending cycle without actionable outcomes.

The HOW of User Interviews, Step by Step

User interviews are indispensable in obtaining genuine user insights, which are crucial for informed decision-making in product development. However, conducting these interviews without a structured approach can lead to several pitfalls including misguided product decisions, biased responses, unprofessional demeanor in front of users, and potentially damaging user relationships.

Types of User Interviews

• Structured Interviews: These are highly scripted and resemble surveys. They’re less favored for in-depth insights as they don’t allow deviation based on respondent feedback.
• Unstructured Interviews: Ideal for experienced researchers, these interviews are conversational and exploratory, potentially lacking focus without skilled handling.
• Semi-Structured Interviews: Recommended for most research needs, this format involves a balance of predetermined questions and the flexibility to explore topics as they arise during the conversation.

Preparing for Interviews

• Define Objectives: Clearly articulate what you want to learn from the interviews. Documenting objectives helps clarify your goals and tailor your questions accordingly.
• Select Participants: Choose a diverse group of users to avoid biased data. For continuous projects, you might start with a small number of interviews, increasing to dozens for larger-scale insights.
• Communication: When reaching out to potential participants, avoid using the term “interview” as it can be intimidating. Opt for softer language such as “We’re conducting some research” or “I’d like to have a chat.”

Crafting the Interview Guide

Question Design — Start with open ended questions!
Initiate the dialogue with broad, open-ended questions that encourage expansive thinking and sharing. As the conversation progresses, introduce more specific queries to delve into the details and clarify any points of interest.

• “Describe a typical day for me.”
• “Can you tell me about a time when you used our product?”
• “Tell me about your most recent experience using our Feature X.”

These questions should set the stage for a rich conversation, providing the participants with the opportunity to share comprehensive and detailed responses.

Transition to Probing Questions

As you gather more context, begin asking probing questions to uncover deeper insights and specific details.

• “What motivated you to choose that particular feature?”
• “Can you explain why you felt that way?”
• “Why do you think that feature stands out to you?”

Do’s and Don’ts of Crafting Questions:

Do:

Formulate questions that are open and broad yet specific enough to elicit focused responses.

Limit the number of questions to maintain clarity and depth in the responses.

Use follow-up questions and probes to explore deeper into the participant’s initial responses.

Don’t:

Pose leading questions that might suggest a particular answer or bias the responses.

Combine multiple inquiries into a single question, which can confuse respondents and dilute their answers.

Overcrowd the interview setting, as too many observers can alter the dynamics and affect the participant’s comfort and candor.

Conducting the Interview

Building Rapport
Begin each interview on a gentle note to make the participants feel at ease. Establish rapport using both verbal cues (tone, pace, volume) and non-verbal cues (body language, facial expressions).

Effective Communication Techniques

• Use mirroring to reflect the participant’s language and body posture, enhancing rapport.
• Allow pauses during the conversation — these moments of silence give participants time to think and provide more thoughtful answers.

Summarize
Keep the discussion focused and productive by summarizing what the participant has shared and ensuring the conversation remains on track.

• “So, what you’re saying is…”
• “Going back to your earlier point about X, could you expand on that?”
• “How does that experience relate to what we discussed earlier?”

Analyzing the Data

Thematic Analysis
Identify and code recurrent themes across the interviews. This method helps in recognizing patterns that may indicate broader user behaviors or sentiments.

Avoid Common Pitfalls in Analysis:

• Do not analyze the data in isolation; involve multiple perspectives to ensure a balanced interpretation.
• Beware of coding every detail — focus on significant and recurrent themes to maintain relevance and manageability.

Reporting Findings

Focused Reporting
Only report findings that are directly relevant to the initial research questions. This ensures clarity and applicability of the insights gained.

Presentation of Findings

• Use quotes and detailed accounts from participants to substantiate the themes identified.
• Clearly articulate how these findings relate to the broader goals of the project, providing actionable insights.

Dovetail: Implement platforms like Dovetail to organize, analyze, and store insights efficiently. Such tools can help synthesize data from multiple interviews, aiding in the identification of patterns and themes.

They offer a free version, but the paid one if highly affordable — $29 User / month billed yearly or $39 billed monthly. Worth every penny!

Nielsen Norman Group — Here’s the User Interview 101 Article that can explain even better than I had + I recommend taking their User Interviews course. True Hands On experience, you can jump right in after the intense 8 hours course.

User Interviews 101

User Interviews | NN/g Training Course

It’s essential to approach each interview with a clear understanding of what you aim to learn, and to tailor your questions to meet these objectives while avoiding biases that could skew the data.

Building rapport and maintaining a focused dialogue throughout the interview process are equally critical in ensuring that participants feel valued and understood, thereby encouraging richer, more meaningful responses.

Finally, the analysis and reporting of findings must be handled with care to accurately reflect the voice of the user and support the strategic directions the organization chooses to pursue.

When done correctly, user interviews can provide a profound level of insight that is instrumental in crafting user-centric products that resonate well with the target audience.

Good Luck and Have Fun! 😼