An overview to qualitative and quantitative research methods in design

Why is getting research more and more important in design? Marc Steen, a research scientist, says: “More and more innovations in the industry are lead by technological developments, rather than by consumers, users need or individual preferences. This so called technology gap brings the risk of creating products and services that people can not, or do not want to use or engage with.” (Steen, 2012) Research in design is “highly contextual and a key principle of a human centered design process by involving users in one or many parts of the design process.” (Steen, 2012) The goal is to understand people’s actions, needs and behaviors, to gain a deeper understanding of the problems, and receive valuable insights for the next design decisions.

Designers have various tools available to do research. From face-to-face interviews, online surveys over to new tools like guerrilla testing. In most cases it is not possible to use all of these research tools in one project, due to budget or time limitations. The key to successful research is to understand the underlying methodologies, to choose the right tool for a project.

What is qualitative research?
Qualitative research is exploratory or interrogative research and tries to get “under the surface”. “The aim is to gather insights into how people live; what they do; how they use things; or what they need in their everyday or professional lives.“ (Government Design Service Manual, 2016)

Interviews are a good example how qualitative research can look like. For designers, they help them to understand the attitudes or mind-sets of a person, by asking for the “Why” and “How”. “If forces the interviewed person to think about motivations and reasons for a certain behavior.” (Barnham, 2015) Qualitative research often happens directly on-site. “It can range from a one-hour face-to-face interview, through following a participant for several days, or even a study over several weeks or months.” (Government Design Service Manual, 2016)

The gained results are rich, detailed insights of a person’s feelings and thinking. The gathered data is unstructured, and can be notes, drawings or even pictures. Qualitative research includes a small selection of participants, based on criteria’s defined by the researcher. The expectation is that this small selection represents a bigger group. This method does not claim that the results are universal right, nor statistical correct or can be reproduced. Qualitative research methods can provide deep insights, and therefore provide the designer with a better understanding for the researched topic.

What is quantitative research?
Quantitative research tries to find answers to concrete questions by generating numbers and facts. “The goal is to establish a ‘representation’ of what consumers do or what consumers think.” (Barnham, 2015)

Surveys are good example for quantitative research. They ask for the “How much” or “How many” by a set of clear and predefined questions. Participants then have to choose which answer (“Yes”, “No”, “Never”, “twice a week”) is the best representation for them. Quantitative research is conducted indirectly or off-site. It is a method that includes large groups of participants to gather relevant data. The results of quantitative research are numbers or facts. In contrast these “facts” can be easily wrong interpreted because of the lack of context. And there is no control if the participant has understood the questions correctly, or what his underlying motivation is.

Quantitative versus qualitative research methods
Let me summarize the key features of both methods: Qualitative research tries to understand the context and has a holistic approach. The researcher does not exactly know what he is looking for. The results are insights in the form of Words, quotes or pictures. The results are a interpretation of the researcher and the data is unstructured. On the other hand, quantitative research tries to classify, count or measure. The researcher knows in advance exactly what he is looking for. The results are measurements, in most cases numbers. Quantitative research has the approach to proof a certain question and is therefore structured.

Qualitative research methods generate insights that are always subjective and an interpretation of the person who is conducting the research. The results are not replicable and not statistical correct, but can provide valuable insights for a human centered design process. On the other hand quantitative research methods are a good instrument a later stage of the process to validate prototypes or measure improvements.

To say that quantitative research methods are therefore objective and qualitative methods are subjective, is an over simplification of both methods and is not correct. It is very important to understand that both methods do not work against each other.

How many people to ask?
Generalization or sampling, is the process of cutting out extremes or so called “outliners” from a research and is a standard in quantitative research. The goal of qualitative methods in contrast, is to look exactly for those “outliners”, because they can “provide a rich, contextualized understanding” (Polit, 2010) and be the most valid source for inspiration in the research process by a “…intensive study of individual cases.” (Polit, 2010)

This leads us to the question of how many observations are necessary to gain a deeper understanding of people’s behavior. In general, it depends on the research method, the approach and the time that is available. Jackob Nielsen, for example, says that five persons in a qualitative study are in most cases enough for a qualitative survey about human centered design questions. (Nielsen, 2012) He argues that the first interviewed person provides often the most insights. The second person will provide similar insights and will add some new insights to the research and so on. Most important is the fact, that asking no people in the design process gives no insights to the designer, and the more persons you add, the insights are not getting linear more.

Mixed methods and hybrid data
“Mixed methods research is, generally speaking, an approach that attempts to consider multiple viewpoints, perspectives, positions, and standpoints. Although mixed methods research is not new, it is a movement that has arisen in response to the currents of qualitative and quantitative research.” (Johnson, 2007)

New inputs devices such as cameras, smartphones, sensors or open accessible public data give designer’s new tools to track and understand people’s real behaviors in real-time, and are constantly connected to the internet. Designers sometimes argue that tracked datasets are soulless and not a valid input. But when we can combine personal insights gained by qualitative research and combine it with quantitative behavioral data, it will enable designers to ask smarter questions in the design process, or to cite Seemann: “Hybrid data, allows us to keep the soul in the data.” (Seemann, 2012)

Mixing both methods, and adding behavioral data from sensors, we can create a new dimension in understanding people better, by analyzing what people actually do and how they behave, instead of asking them on how they think and feel. Because what people say, think, and feel can be contradictory to what they actually do. It is not that people are being dishonest, in fact, quite the opposite, they are giving as honest answer as they can. (Cooper-Wright, 2015)

By allowing designers to toggle between qualitative and quantitative methods during rapid prototyping and research synthesis …, hybrid data points to the future of smart insight generation. (Seemann, 2012)

Bibliography
Barnham, C. (2015) Quantitative and qualitative research: perceptual foundations, International Journal of Market Research, vol. 57, no. 6, pp. 837.

Cooper-Wright, M. (2015) Are You a Good Driver? How Designers Use Data to Get to the Truth Available at: https://medium.com/design-x-data/are-you-a-good-driver-how-designers-use-data-to-get-to-the-truth-3c534fcaf9d2#.75buosl2w (Accessed: 20 March 2016).

GOV.UK (2016) Ethnographic research: Getting input into products and services Available at: https://www.gov.uk/service-manual/user-centred-design/user-research/ethnographic-research.html (Accessed: 20 March 2016).

Johnson, R.B., Onwuegbuzie, A.J. & Turner, L.A. (2007) Toward a Definition of Mixed Methods Research, Journal of Mixed Methods Research, vol. 1, no. 2, pp. 112–133.

Miles, M.B., Huberman, A.M. & Saldaña, J. (2013) Qualitative data analysis: a methods sourcebook, 3rd edn., SAGE: Los Angeles, pp. 40.

Nielsen, J. (2012) Why You Only Need to Test with 5 Users Available at: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ (Accessed: 20 March 2016).

Polit, D.F. & Beck, C.T. (2010) Generalization in quantitative and qualitative research: Myths and strategies, International Journal of Nursing Studies, vol. 47, no. 11, pp. 1451–1458.

Rohrer, C. (2014) When to Use Which User-Experience Research Methods Available at: https://www.nngroup.com/articles/which-ux-research-methods/ (Accessed: 23 March 2016).

Seemann, J. (2012) Hybrid Insights: Where the Quantitative Meets the Qualitative Available at: https://www.ideo.com/images/uploads/news/pdfs/pp_56-61_HybridInsights_RotFall12_spreads.pdf (Accessed: 20 March 2016).

Steen, M. (2012) Human-Centered Design as a Fragile Encounter, Design Issues, vol. 28, no. 1, pp. 72–80.