This article is part 1 of a series concerning the relationship between UX and research. Part 1 will argue for the scientific acknowledgement of UX methodology and describe how research integrates into the design process. Part 2 can be found here.
I work for a company of about 800 employees. Over the past year, we’ve been undergoing a transition from using legacy enterprise ideology into making data-driven decisions driven by UX methodology. That’s a huge undertaking to align everyone on what UX even is, let alone arming them with the tools to significantly change the way they look at business.
I came on board as one of 4 people to help bring this company into the digital age. Over the span of a few months, my team had quickly grown from a few designers and front-end developers into the addition of UX researchers and back-end engineers. One of the things we’re trying to communicate to the rest of the organization is not only how to conduct research, but how to analyze, synthesize, and interpret data. Even then, we needed to illustrate how data integrates into high-level strategy and design decisions.
At first, our team found the task of communicating this concept rather easy. It doesn’t take much to convey how data equates to dollars and cents. In fact, that’s speaking their language. And what’s even better — news travels fast. It wasn’t long before saying we were a data-driven company became everyone believing in it. To a legacy organization, messaging is key in promoting alignment. However, that was just the first step.
The second consisted of all employees taking initiative in adopting this concept into their workflow. We were initially under the impression, or rather the assumption, that a data-driven process was self-explanatory. We were thrilled to see employees questioning their day-to-day decisions to understand if they were making informed calls. This prompted collective action, which is exactly what we wanted to see.
We quickly learned that too much autonomy became dangerous once some employees exclaimed they were now researchers and designers. A lack of alignment persisted amongst sources of truth and in how data was both analyzed and interpreted. We had initially envisioned our team to lead by example in this regard, however we realized that we had to take a step back and start with communicating the basics.
This is why defining research and data analysis is critical to the success of any organization. More importantly, it is imperative to emphasize the prominence of research throughout the UX process to both clients and employees alike. In order for a company to be truly UX-centered and data driven, it must understand the role of how research integrates across all departments of the organization alongside its role in UX.
This experience quickly illuminated the fact that that we needed to improve how we, as designers, researchers, and engineers, defined and discussed UX to both those within and outside of the design community. Therefore, I am prompted to argue for UX as a methodology, rather than what some might believe to be a new internet buzzword.
We have come to understand that the mainstream message beyond Silicon Valley and the design community surrounding the definition of user experience is vastly misconstrued. It’s built upon a platform of assumptions and misunderstanding where many unfamiliar to UX only see a part without understanding the whole.
Fundamentally, UX needs to be recognized as a scientific practice that is being utilized to lead contemporary business models. This might sound like a foreign concept to some, but if we take a look at how scientific research integrates into UX and business, then we can understand how this makes sense.
Science is a way of thinking much more than it is a body of knowledge. — Carl Sagan
Building Scientific Design
Let me begin by saying that UX researchers, designers, and engineers are scientists. Together they ask questions, formulate hypotheses, test, validate their claims, and make sense of the data they’ve received. They’re not cooped up in a lab away from the civilized world, but are instead out in the field attempting to learn and improve upon their lab-based creations. They’re sociologists and cognitive psychologists interested in understanding human behavior and influencing decisions. They observe problems before formulating hypotheses and research questions in order to conduct research and solve them. Their methods are dynamic, often unconventional, and relative to the problem they’re trying to solve. Most of all, they challenge the status quo in search for holistic solutions.
Many might believe that science and creativity are mutually exclusive concepts. This notion is a stereotype. The scientific method is not as bland as it might seem. The process of identifying a phenomena of interest, discerning the problems, designing the method of experimentation, and discussing the relevance of results is both highly creative and strategic. There is a reason as to why scientists say that an experiment needs to be designed. Science is as creative as design is scientific. Albert Einstein communicates this integration well.
The formulation of a problem is often more essential than its solution, which may merely be a matter of mathematical or experimental skill. To raise new questions, new possibilities, to regard old problems from a new angle, requires creative imagination and marks real advance in science.
After a certain high level of technical skill is achieved, science and art tend to coalesce in esthetics, plasticity, and form. The greatest scientists are artists as well.— Albert Einstein
One can even argue that beyond the human-centered approach, UX methodology is derived from science. From an elementary point of view, UX methodology follows the tenets of logical positivism, a philosophical mode of thought where knowledge is ceded from rational conclusions that are verified by observable facts. In other words, any logical statement is only valid if supported empirically. And empirical evidence is any conclusion deduced by scientific experimentation.
Contemporary research is inherently empirical. Scientists from both the hard sciences (mathematics, physics, biology, etc.) and social sciences (sociology, political science, economics, etc.) are practicing research with the same understanding. Empiricism lies at the heart of UX.
Therefore, any decision or assumption that design teams make under this paradigm needs the existence of data to support the claim.
This is what makes the agile method so successful. You research, design, launch, test, and iterate. Then you test, launch, and iterate again. And again and again and again and again and again. At Stanford’s d.school, students are taught this framework as part of their design education. This concept not only prepares them for the business of product design, but seeks to instill this process as a fundamental aspect of the design ideology. (Of course, there are exceptions to the rule in certain instances. But for the most part, empirical data is king.)
A Few Methods To The Madness
Now that we’ve discussed UX as a scientific process, I want to explain exactly how data is integrated into business practice by driving design decisions. Whether at the ideation or iteration stage of building a product, designers need to collect data utilizing quantitative, qualitative, or mixed methods research.
A business will see trends in numerical data relating to traffic, engagement, retention, and conversion. These are all concepts that exist to indicate the performance of a particular product and its direct relationship to revenue. Quantitative research in the form of statistical analysis serves as a means of indicating what exactly is happening to a product.
When collected over a period of time, longitudinal trends are utilized by designers during the evaluation of a product’s problem areas, or friction points for users, to evaluate what needs to be solved for. Statistical data also provides insight into the general health of a product and acts as a supporting platform for the emergence of KPIs. This is where UX teams see a north star and decide to move towards it.
On the other end of the spectrum, qualitative data is best used to understand why a product might be performing as it is. While quantitative data presents a high-level understanding of a particular phenomena or trend, qualitative data in the form of interviews, focus groups, ethnography, and usability testing (to name a few), investigates deeper into the problem.
What are user’s motivations for using the product and what does their behavior look like when doing so? Why has conversion decreased? Why are users not responding to this call to action?
These types of questions lead to insightful discoveries that influence product strategy. In other words, while quantitative data indicates both the successes and problems of a product, qualitative data can help demonstrate how we might need to solve for those problems.
Mixed methods are our bread and butter. It’s what makes UX absolutely thrive. This combination of both qualitative and quantitative research can be utilized to increase the validity of our findings and present a holistic analysis of observed phenomena. An increase in validity places UX one step closer to empirical evidence and presents an increase in confidence when moving forward throughout the design process.
For example, we might look at Google Analytics to see how traffic patterns for our product have shifted over the last six months before conducting user interviews to validate or invalidate those findings. This can also be understood in terms of an if, then statement.
If traffic has decreased by x%, then x could be why it’s doing so.
A mixed methods approach describes an entire story by questioning the what and responding with the why. In order to construct solutions as a result of mixed methods, a clear narrative needs to be spoken from both the users and product itself. However, it is critical to note that quantitative and qualitative approaches can answer both questions interchangeably. Research remains far from being categorized in a linear method and can begin at any point.
Nothing Moves From Point A To Point B
While the agile method is repetitive and iterative in nature, UX methodology remains inherently dynamic as well. Just as a product remains under cycles of testing and iteration post-launch, research continues to follow suit. New information can be discovered at any stage of development and reflected into the product.
Nevertheless, it remains critical for research to continue achieving breakthroughs in order to maintain success. As markets and users continue to shift behaviors, preferences, and usability, research needs to continue influencing a product’s strategic direction. When uncovering and evaluating problem areas, designers and researchers utilize both inductive and deductive approaches.
This approach begins with the observation of a problem and subsequently moves towards understanding why it may be happening. Inductive reasoning is most commonly used during the primary research phase of building a product. It first begins without any empirical evidence and strictly relies upon observation. Teams may use interviews, surveys, and field research to gather data. At this stage, teams begin to ask questions surrounding a phenomenon without any assumptions about the cause and work their way up to proposed solutions. This is typically understood as the bottom-up approach.
Or as Drake would say… 😋
A deductive approach is exactly the opposite. In this case, teams work from existing empirical data, devise a hypothesis, and test it. This typically comes much later in the design process once testing and iteration is taking place. At this stage, techniques such as eye-tracking, A/B testing, and prototype testing are used. Traditionally, the scientific method calls for theory to drive deductive reasoning. A hypothesis is derived from a theoretical framework and tested with the intention of building upon that theory. This is also known as the top-down approach.
The final method of reasoning is supported by inference. Abductive reasoning follows the best explanation one may have as a result of observation and testing. Similar to inductive reasoning, abduction begins with a set of data that can be utilized to infer another set of data. A simple abduction can be understood from the following example:
You wake up in the morning, and you head downstairs. In the kitchen there’s a plate on the table and a bowl with a little milk left in it. You abduce that the explanation for this is that your housemate awoke before you, had their breakfast, and left.
User Experience Magazine defines the use of abductive reasoning within the UX process as being a combination of both inductive and deductive approaches. Under this presumption, the use of both forms aid to construct a holistic research approach that collects data from users under a mixed methods paradigm. Abduction in this sense is being used to generate the best conclusion possible after numerous research methods have been used.
Isn’t this all so exciting?! Now on to wrapping this up. 😀
UX serves as the overarching framework that houses research, strategy, copywriting, UI, and development. The entire process holistically utilizes a scientific approach to maintain the balance between improving the user experience while also fulfilling business objectives (such as increasing revenue). Teams need to have an understanding of how all of these things work together in order to be successful.
We’ve seen that a synthesis of mixed approaches for both testing and reasoning embodies a complete methodology. Although UX is heavily developed and understood in practice, improvements need to be made in terms of how UX is both defined and communicated outside of the community. Designers need to begin acknowledging how integrated UX is as a business model and scientific procedure. While UI and aesthetic appeal are simply one facet of the methodology, it should not dominate this definition.
As UX continues to progress, design education needs to begin addressing these concepts and replace existing assumptions surrounding the nature of UX. For a field that has been so niche in nature, we need to better define how it works to everyone else. But if we can first start to recognize UX as a scientific practice that’s flourishing with creativity and innovation, then that’s a huge win!