Research Questions Are Not Interview Questions

Erika Hall
Jan 10, 2019 · 6 min read

(You can’t just ask people what you want to know. Sorry.)

The most significant source of confusion in design research is the difference between research questions and interview questions. This confusion costs time and money and leads to a lot of managers saying that they tried doing research that one time and nothing useful emerged.

(It’s even worse than how the terms “user research” and “user testing” are thrown around interchangeably. They are not interchangeable. The former is the process of learning about the people who are the intended users of a system, and the latter is evaluating the system itself against established criteria by having people interact with it. And for the record “user testing” isn’t really a thing because you aren’t testing users, you are testing the usability, or other qualities, of a system. Intentional phrasing matters.)

Your research question and how you phrase it determines the success and utility of everything that follows. If you start with a bad question, or the wrong question, you won’t end up with a useful answer. We understand this in daily life, but talking about research in a business context seems to short-circuit common sense. Everyone is too worried about looking smart in front of each other.

Research Questions

  1. It is possible to answer the question using the techniques and methods available to you
  2. It’s possible (but not guaranteed) that you can arrive at an answer with a sufficient degree of confidence to base decisions on what you’ve learned

If your question is too general or just beyond your means to answer, it is not a good research question. “How’s the air on Mars?” may be a practical question for Elon Musk to answer to meet his goals, but it isn’t for most of us.

Research Question Examples

  • How do recent college graduates decide what and when to eat?
  • What are the greatest barriers to saving money for Canadians in their 30s?
  • In what US cities do residents spend the highest proportion of their income on vacation travel?
  • How well does the latest version of the marketing website answer prospective customers’ questions?
  • How does Company X create their product roadmaps?

Only after you have identified your research questions can you select the best way to answer the question. A lot of people get this wrong and pick the research activity first.

NO: “We’re going to run a survey. What should we ask?”

NO: “We’re testing a prototype with non-customers on Friday. What do we need to know?”

YES: “We need to understand the factors that influence how our current customers select and view television programming, so we are going to recruit and interview current customers in their homes.”

You won’t even know whether doing interviews is the right thing until after you’ve written your research question. It might be more effective to read existing literature, or observe people out in the world, or do a competitive analysis instead.

Maybe the best way to find out whether anyone will develop a habit around your is app to learn what gets people to floss their teeth. Maybe the best way to understand your prospective customers is to usability test a competitive service. Resist the urge to narrow your research focus too quickly or you will miss opportunities to learn what you didn’t think to ask. Resist the temptation to ask your questions directly or you will be basing decisions on total fabrications.

Special comment about organizational research

In addition to the wider business landscape and social context, your research questions must include the organization in which you’re working. Organizations are the social context in which design and product decision-making happens. If you don’t understand how people make decisions in your organization, you will never be able to influence them.

Example organizational research questions

NO: How shiny does our report need to be to impress the leadership?

YES: Who makes product decisions, what incentives influence them, and what sources of information do they trust?

Facts don’t change minds. Reports are easy to ignore. Reality often contradicts the wishes of authority. It doesn’t matter how good your data is if you haven’t done the groundwork to ensure an evidence-based framework for making decisions is in place before doing the work of gathering other evidence.

Don’t be afraid to step back from your comfortable assumptions and start at a high level. When you consider what you need to know to solve the whole design or business problem, it probably includes far more than user behavior. You don’t need that much more time, but you will need a lot of courage.

Quantitative vs Qualitative

A lot of bad research results from a mismatch between question and method, usually because people spend more time worrying about the activity (surveys, testing, interviews) than about forming a good question. Even more bad research is designed specifically to provide support for an existing solution. (That’s why it’s called “validation”.)

Ethnographic interviews are a qualitative method.

Interview Questions

Research Question Example

YES: How do families with school-age children decide how to spend money on vacations?

Interview Question Example

YES: “Walk me through your last vacation from planning it until when you arrived back home.”

That might be all you have to say to learn about how people choose a destination, how they budget and spend their money, and how they share information and make recommendations to their friends.

Some research questions are completely unanswerable if you pose them directly, especially if they are about motivation or sensitive topics such as health and money.

You cannot ask people to predict anything or to remember something that was too far in the past. No one can tell you how likely they are to do anything in the future. And humans are way better at fabricating answers than at remembering past events accurately.

When surveys are written without regard to context and with untested questions, it’s really easy to gather a bunch of garbage data based on unanswerable questions.

Bad Interview/Survey Question Example

NO: “On a scale of 1–10 how likely are you to buy a new smartphone in the next 6 months?”

No one can answer that. Do not ask anything like that of anyone ever. This is an extreme example, but I’ve seen similar in the wild many times.

Go forth with rigor

Do you need some professional help with your organization’s approach to making decisions, but aren’t ready to dive into a full workshop or consulting engagement?

For a short time, I’m offering a 1-hour live Q&A with your team. I guarantee it will be the best Zoom of your day. Reserve your spot now!

Mule Design Studio

Designed to work.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store