When to apply qualitative and quantitative research

User research is a non-negotiable component of experience design. It’s what gives us consumer insight which in turn helps us make informed decisions about design. There are two main types of user research: qualitative and quantitative and of course there are pros and cons associated with both. When to deploy either-or depends very much on the design decision you need to make.

Qualitative research methods are used to gather feedback and insights from a relatively small group of users. In general, we describe qualitative research as answering the; How, When, Where and Why, of user behaviour. Examples of qualitative methods include: unstructured or semi-structured interviews, ethnography, diary studies, and focus groups and co-creation sessions.

Capabilities

• Explore or validate concepts/designs with a small sample (approx. 12–50 participants)
• Observe immediate reactions and emotional responses to an experience, including tone of voice, facial expressions and body language
• Uncover unexpected pain-points or opportunities to add value through live probing

Limitations

• Sample size is limited, therefore lacking the confidence of numbers
• Sample obtained is not (statistically) representative of entire user population
• Unstructured methods can lead to inconsistent data capture

About quantitative research

Quantitative research methods are also used to gather insights but the size and range of the user group can be expanded. This type of research focuses more on collecting user “data” as opposed to just insight. In general, we describe quantitative research as answering the “who” and “what” of user behaviour. Examples of quantitative methods include: structured, unmoderated testing, surveys, and questionnaires.

Capabilities

• Explore or validate concepts and designs with a large sample (approx. 50–1000+ participants)
• Capture behavioural data for task-based research, including task completion times, clickpaths, heatmaps, and success or failure scores
• Measure satisfaction and usability with standard measures (i.e. System Usability Scale, Net Promoter Score)

Limitations

• Unable to probe participants in the moment on their responses or observed behavior
• Lacks moderation; therefore the stimulus presented must be functional for use without assistance from a moderator

Choosing the right method to inform design decisions

Research methods are not interchangeable and have specific objectives which they are most suitable for. Choosing the most applicable method will depend on what design decisions need to be made for the creation or refinement of a design.

As a rule of thumb, use qualitative research methods for exploring ideas, designs, or processes which do not require concrete hypotheses or structured, statistically valid feedback. We have found that qualitative methods are paritcularly useful for serendipidous discovery, and often provide more in-depth insights.

You can use quantitative research methods such as A/B testing for validating or choosing a design based on user satisfaction scores, perceived usability measures, and/or task performance. You will find the insight will be more superficial, but the data is statistically valid and can be generalised to the entire user population.

A combination of qualitative and quantitative research is typically best for most design projects, if budget allows. By using both methods you can achieve a deeper level of insight through the exploratory nature of the research in addition to statistical evidence to support your design decisions. Using qualitative and quantitative research together answers to the; Who, What, When, Where, Why, and How, of user behaviours and experiences.

Originally published at www.foolproof.co.uk.