User Research Questions

Photo by Mapbox on Unsplash

In the realm of user research, the analogy of Egypt’s Valley of the Kings serves as a poignant reminder: just as an archaeologist’s success hinges on excavating the right spot, a UX researcher’s triumph lies in asking the right research questions. Much like Howard Carter’s meticulous search for King Tutankhamen’s tomb, crafting the perfect question is an art that can reveal a treasure trove of valuable insights into user behavior and experience.

In this chapter

• What is a user research question?
• Benefits of asking good research questions
• Examples of good research questions
• How to write an effective research question
• Mixed methods research questions

What is a User Research Question?

A user research question serves as the fundamental compass guiding a research study, distinct from user interview questions. Unlike interview questions, research questions shape the entire trajectory of a study, determining methodologies, insights, and subsequent decisions.

Characteristics of a Good Research Question

1. Specificity: The question should be specific, allowing for a focused investigation. For instance, refining “How do university students use Linkedin?” to “How do American university students use Linkedin for career-related research?” adds specificity.
2. Practicality: Ensure that the question is practical and can be feasibly answered within the constraints of available time and resources. Avoid overly ambitious inquiries, opting for questions aligned with practical research goals.
3. Actionability: The question’s insights should lead to actionable outcomes, influencing decisions or prompting changes. Avoid purely informational queries that lack relevance to tangible decision-making.

Questions starting with “how much” or “how many” are suited for quantitative methods, while those beginning with “how,” “what,” or “why” align more with qualitative methods.

Initiate research with questions to ensure an investigative approach focused on real problems rather than validating pre-existing assumptions or biases.

Types of Research Questions

Effectively framing research questions involves various types that cater to specific investigative goals:

1. Descriptive Research Questions

Objective: To describe variables, behaviors, or phenomena.
Examples: What percentage of Canadian college students use public transportation to get to campus? or What is the demographic composition of users engaging with our mobile app?

2. Causal Research Questions

Objective: To assess the relationship between two or more variables.
Examples: What effect does in-app purchases have on the overall revenue of our platform? or How does the introduction of a new feature influence user engagement?

3. Comparative Research Questions

Objective: To evaluate differences between two or more groups concerning specific variables.
Examples: How does screen time vary between male and female Americans under the age of 18? or What are the contrasting user preferences for feature A and feature B?

4. Key Considerations

Correct Definition: The effectiveness of each research question depends on its accurate definition, ensuring specificity, practicality, and actionability.

Matching with Research Method: Align the research question with an appropriate research method to derive meaningful insights. For instance, qualitative methods may suit descriptive questions, while causal questions might require quantitative approaches.

Synthesis for Stakeholders: Synthesise research findings in a way that is comprehensible and beneficial for stakeholders, facilitating informed decision-making.

While each type of research question holds value, their efficacy lies in the precision of their definition, compatibility with chosen research methods, and the synthesis of results into actionable insights for stakeholders.

Research Question vs. Hypothesis

While related, “research questions” and “hypotheses” serve distinct roles in the research process:

Research Question

Definition: A focused inquiry that lays the groundwork for the research study.
Role: Establishes the foundation, guiding the research project and influencing method selection, participant recruitment, and subsequent actions.
Example: What is the effect of in-app purchases on overall revenue?

Hypothesis

Definition: A testable assumption predicting the answer to the research question.
Role: Represents the anticipated outcome and serves as a basis for testing during the research study.
Example: If we offer in-app purchases, then we’ll increase revenue by X%.

Typically, a research question precedes a hypothesis, except in cases where a substantial body of research informs the hypothesis post-product launch.

Research Questions vs. Interview Questions

Distinguishing between research questions and interview questions is vital for effective design research. Erika Hall, Co-Founder of Mule Design, highlights this key difference in her article: ‘Research Questions Are Not Interview Questions’

The most significant source of confusion in design research is the difference between research questions and interview questions. This confusion costs time and money and leads to a lot of managers saying that they tried doing research that one time and nothing useful emerged.

User Research Questions

• Core questions framing learning objectives for individual research projects.
• Outlined in the research plan, guiding the study, influencing method selection, participant recruitment, and actions taken based on findings.

User Interview Questions

• Questions posed to participants in 1–1 interviews during a research project.
• Informed by the core research question but broader and more open-ended to avoid leading participants.

Mapping Interview Questions to Research Questions

Avoid using research questions as interview questions to prevent biased answers influenced by participants’ knowledge of the study’s goals.

Example:

• Research Question: How do single 25–35 year olds choose where to go out on a Friday night?
• Interview Question: Walk me through your last Friday night outing, beginning with when you started planning the outing.

A research question is a focused inquiry setting the foundation for the study. A hypothesis is a testable assumption predicting the answer to the research question.

Examples of Good User Research Questions

Qualitative Research Questions

1. How well do our support pages answer customers’ questions about adding a new credit card to their account?
   This question aims to evaluate the effectiveness of support pages regarding a specific user task.
2. How do families with newborn babies choose which brand of diapers to purchase?
   Investigates the decision-making process of a specific user group regarding a common product.
3. Why are so many people abandoning their shopping cart?
   Explores the reasons behind a common user behaviour (cart abandonment) to identify potential pain points.
4. What are the primary motivating factors behind the decision to purchase first-aid kits?
   Seeks to understand the motivations influencing users in a specific purchase decision.
5. What tools do freelance writers use to keep track of their schedules?
   Investigates the tools and methods used by a specific user group for managing their work schedules.
6. Which apps do women and non-binary folks who are looking to date other women and non-binary folks use to meet potential partners?
   Focuses on the app preferences of a specific demographic for a particular purpose.

Quantitative Research Questions

1. How frequently do adult European skiers replace their ski boots?
   Aims to quantify a specific behaviour (frequency of ski boot replacement) among a target user group.
2. What percentage of our customers prefer the mobile app to the website browser?
   Quantifies user preferences between different platforms to inform development priorities.
3. How much do families with teenage children spend on movie tickets in the United States?
   Provides numerical insights into the spending habits of a specific demographic for a particular activity.
4. How often do working Millennials check their email per day?
   Quantifies a common behaviour (email checking) among a specific demographic.
5. To what extent does alcohol use affect college students’ academic performance?
   Aims to measure the impact of a variable (alcohol use) on a specific outcome (academic performance).
6. What proportion of British men and women ages 25–35 use calorie tracking apps?
   Quantifies the usage of a specific type of app among a defined demographic group.

Examples of User Interview Questions

Research Question

What are the primary motivating factors behind the decision to purchase first-aid kits?

Interview Questions

1. Walk me through the last time you purchased a first-aid kit.
   Encourages the participant to provide a detailed narrative of a recent purchase experience.
2. Which brands did you consider when buying a first-aid kit?
   Seeks information on the brands considered, providing insights into the participant’s decision-making process.
3. Tell me about a time when you needed a first-aid kit and didn’t have one on hand.
   Explores real-life situations to understand the participant’s experiences and potential pain points.
4. How often do you use your first-aid kit?
   Aims to quantify the frequency of usage, providing insights into the practicality and relevance of the kit.
5. What do you appreciate most about your current first-aid kit?
   Focuses on positive aspects, identifying features or qualities that users value in their current product.
6. Is there anything missing from your first-aid kit that you wish was included?
   Encourages participants to discuss potential improvements or additions, revealing unmet needs or desires.

These interview questions are designed to gather rich, qualitative insights into the motivations and behaviours surrounding the purchase and use of first-aid kits. They are open-ended and allow participants to share their perspectives and experiences freely.

How to Write an Effective Research Question

Crafting effective research questions is a crucial step in conducting meaningful user research. Here’s a guide on how to write an effective research question:

1. Identify Your Research Goals

• Understand the pain points, challenges, and priorities within your team.
• Align research questions with high-level company goals and team priorities.
• Use a Decision-Driven Research approach to link research questions to specific decisions.

By starting with the decisions your team needs to make, you can reverse-engineer your research questions to address the gaps in the information your team needs to make those decisions.

2. Scope Existing Evidence

• Evaluate what is already known about your research goal.
• Review existing customer feedback, analytics dashboards, and other relevant sources.
• Interview stakeholders to gather insights and experiences.

Sometimes, you might find that your question has already been answered — and you’ll save yourself from duplicate work. Other times, you’ll find information that helps you hone, pivot, or revise your question, setting the stage for more impactful research.

3. Create Your Research Question(s)

• Develop a list of potential research questions based on the decisions and information gaps.
• Ensure that questions are specific, practical, and actionable.
• Consider the design cycle, available time and resources, and the intended users of the research.

You might need to take vague, high-concept requests from stakeholders and refine them into concrete, answerable questions.

What do you mean, ‘the site has become better’? What scenarios are important to us? What metrics are important to us? What changes should occur in the physical world as a result of launching a new website/product? Depending on the answer, the methods of verification will be different.

Research questions should NOT be easily answered with a Google search, already answered by previous research, or asked only to validate biases.

4. Narrow Your Focus, Then Complete Your Research Plan

• Limit the core research questions to 3–4 per study to maintain focus.
• If you have more questions, consider breaking the research into multiple phases or projects.
• Select the best research method to answer the chosen questions and create a comprehensive UX research plan.

Many studies can explore multiple questions at once, but no study can effectively answer every question you have.

This systematic approach ensures that your research questions are well-defined, aligned with organisational goals, and capable of providing actionable insights. Following these steps will lead to more impactful and targeted user research.

In a Nutshell

• Ask Better Questions, Get Better Answers.
• Craft specific, practical, and actionable user research questions.
• These questions guide your entire research project, from methods to insights.
• A well-defined question is the key to meaningful and impactful user research.