A psychological view for UX Metrics
As a psychology student currently working in the technology industry, I believe psychology can offer insights into how to tackle common UX problems. The primary aim of this article is to encourage the integration of knowledge from the psychology and design fields, bridging the gap between the two. By doing so, we can facilitate the development of more advanced research solutions.
When working with User Experience, we are often faced with the question of whether a proposed solution actually brings about positive changes from the user’s perspective. How can we determine whether the user feels that solution A is easier than B? Moreover, how can we compare if our users are satisfied with a new feature?
There are many indirect methods to measure the success of products, such as conversion rate, the number of new users we have gained with a new release, and so on. All of these are usually more related to the product itself. However, one major factor in user experience is that we are trying to figure out how the product we produce makes people feel. So we may ask ourselves: Is there any way of measuring feelings or any other psychological processes?
A glimpse from inside
Neuroscience has now developed in such a way that we have many methods of trying to understand what’s happening inside our brains. MRI (Magnetic Resonance Imaging) machines provide us with a glimpse of what feelings may look like by showing us the electrical patterns in the brain associated with them. This is important to notice. What we see isn’t the feelings themselves, but rather an electrical pattern of the brain that occurs when people feel something. So, when we learn about this, we can recognize and identify the context and behavior of the individual when these feelings (or patterns) occur.
The point is that feelings are a big deal when it comes to User Experience, and they can’t be directly observed. However, it doesn’t mean that we can’t attempt to understand what’s happening in people’s minds. In a certain way, what we may observe from feelings is their silhouette, and from it, we can attempt to infer their true nature. You may be thinking, ‘Okay, but usually I don’t have an MRI machine lying around.’ Don’t worry, there is more to explore.
Trying to observe a feeling it's like trying recognise a flower only by looking to its shadow. We will never know its true shape and color, but we can learn how to make a good guess.
So what are Emotions?
Finally, it's time for us to try to understand better what emotions are before continuing our exploration of how to measure it. There are many theories and there's one that I am partially fond of. A smart neuroscientist called António Damásio understands emotions as a very complex response that has cognitive, behavior and physiological as its components to a stimulus. If you would like to know more, I strongly recommend this book. In short, emotions can be understood as a information given to a stimulus that has a response that won't be restricted to the mind (as a thought), but the body shall manifest the emotion as well.
There are many different physiological behaviors that we can measure that could give us an insight into the mind. For instance, when watching a movie we often have different emotions being elicited, so our bodies react in different ways, we can get agitated, cry, smile, our heartbeat rate or breathing frequency can increase suddenly. All this parameters could be use to infer emotional activity based on its intensity and quality.
Investigating Cognitive aspects
Have you ever found yourself asking why our pupils dilate when we see someone we’re in love with? Well, there are some environmental stimuli that can affect the diameter of our pupils. For example, when there’s a lot of light, they tend to get smaller, and when we are in the dark, they tend to get bigger so that more light can be absorbed by our retina.
Does this mean that somehow our love ones are "dark"? That doesn't make much sense. The explanation for this may be found in the research of another intelligent guy. Daniel Kahneman has a theory that we have two distinct ways of using our brain, two systems. In short, thinking takes a lot of energy and we evolved to be as economical as possible about it. To manage this we have a system know as Fast, witch we use it most of the time, like when we are in a conversation with friends, doing the dishes, or driving on a relatively quiet street. It isn't very bright, but achieves its goal of providing a simple and fast solution to everyday problems. On the other hand, we have a system know as Slow, summoned when "heavy duty" thinking is needed. We use it when there is a task that demands us to use most of our cognitive resources to complete it, like solving a puzzle. Slow is very powerful, but also it's very costly for our brain (so we usually prefer to use Fast most of the time). That's why we often feel exhaust after doing exams or a hard days work. More about it, you can find on this another amazing book.
There is an interesting phenomenon related to the Slow system: when we use it, our pupils tend to dilate. In fact, Kahneman discovered that the dilation of our pupils could be used to measure the intensity of cognitive work. Therefore, when we see someone we are in love with, we often dedicate a lot of cognitive resources to deal with the situation, leading to an increase in our pupils’ diameter (we also get sweaty hand and butterflies on our stomach). This response is not unique to love, as our pupils can also dilate in response to other emotionally charged situations or when we are faced with a challenging task that demands more cognitive effort. Along with other physiological measurements, pupil dilation can be used to infer emotional and cognitive states, providing valuable insights into the user experience.
The important thing to note here is that this observation can also be applied in different contexts, such as in UX research. For instance, in an A/B test experiment for a new app feature’s usability, this method can be used to infer which solution demands less cognitive resources from the user. The more a user needs to think to use a feature, the poorer its usability seems to perform. However, it’s important to remember that there are some other ways to investigate how thoughts and emotions can be measured, and simply pointing a camera at a user’s eye may not be the most effective or ethical approach. My point here is to show that are some ways in which thoughts may be indirectly observed.
Going deeper
Another important concept to be understood is that the meaning of emotions is socially constructed, which means that they are strongly related to cultural and other social aspects. Different cultures and societies have different beliefs, values, and norms that shape how emotions are expressed, interpreted, and experienced. For instance, while some cultures may encourage expressing emotions openly, others may value emotional restraint and self-control. Understanding the cultural and social context of emotions is crucial for designing products and services that are sensitive to the needs and expectations of diverse users.
This can be easily identified when we observe how different cultures express frustration or happiness. For example, think about the differences between a Japanese and an Italian person expressing frustration. This exercise helps us understand that the social behavior of an individual can give us clues about how they feel. This assumption changes the game of how we can figure out what and how people feel and act when they are immersed in performing a specific task. It is important to consider cultural and social differences when designing a user experience, as what works well in one culture may not work as effectively in another.
Let’s recap: Although feelings cannot be directly observed, they influence our behavior in a particular way that is related to the social and historical definition of emotions. This information can be extremely helpful in constructing more appropriate surveys that attempt to capture a latent variable through their questions.
Measuring the invisible
Measuring abstract concepts (constructs) is a common challenge in psychological research. To conduct studies, we often use surveys to measure variables that can’t be directly observed. Let’s take depression as an example. Although we understand that depression is a complex mental illness with specific diagnostic criteria, researchers often use measures of self-reported depressive symptoms to compare different groups or examine relationships with other variables. Because we know that depression is multidimensional and has distinct factors, we can create questions or seek to observe what we believe is related to this construct.
Using the depression diagram presented above, we can create a survey consisting of questions that ask individuals how much they relate to it. For example, questions such as “Do you often wake up feeling tired?”, “Do you have difficulty sleeping?”, “Do you feel that life is not exciting anymore?” and “Do you frequently question the value of life?” can be directed toward the objective of measuring factors related to depression. By measuring the answers to these questions (a direct observation), we can assume that we are measuring this larger construct of depression.
Let’s embrace this idea for now. The truth is that psychology heavily relies on robust statistics to validate what are initially just assumptions. With techniques such as factor analysis, we can gather evidence of how well (or how poorly) we are measuring what we intend to measure, and discuss a margin of acceptable uncertainty for our study. However, the main goal of this article is not to delve into the details of statistical analysis. I’ll soon write a dedicated Medium post that discusses applied statistics in psychology (and design). If you’re curious and want to learn more, I recommend reading this paper.
Usually, when measuring responses to questionnaires, a Likert scale is used with a standard unit of measurement for each question. People are asked to rate how much they agree or disagree with each question using a scale of 1–5 (or 1–7). The instruction is usually to pick 1 if they do not agree with the statement at all and 5 if they strongly agree with the statement. Intermediate points on the scale can be used to indicate varying levels of agreement or disagreement.
Where does this take us? We should now be able to propose appropriate survey questions and measure responses in a standardized way. Remember the experiment where we wanted to understand the user experience for a new feature of our app? We could ask our users to complete a survey and analyze the data afterwards.
The items on our questionnaire would probably look somewhat like the list in the diagram above. The score for each factor would be the sum of the scores of each question that represents it. Therefore, we would have scores that represent the satisfaction level for the new app feature for each factor that we assume is related to it. Additionally, we could also sum the scores for each factor, so we have a unified measure for our construct.
And so, we have accomplished our goal of measuring a feeling indirectly by proposing a questionnaire. However, we must acknowledge that there are limitations to our approach since it is mostly theoretical, and reality may not always behave as we expect. Nevertheless, for the time being, this will suffice.
It’s important to keep in mind that the construct model for your research heavily depends on your goals and what you want to measure. This means that questionnaires cannot be created and applied without caution. You need to carefully consider what you want to measure, why you want to measure it, and how to measure it, since cultural context also can directly impact on this process. A great way to avoid mistakes is to work in a group, where you can discuss and get feedback to ensure that you are on the right track.
All about the User
We can use very creative methods to get a glimpse of what’s inside people’s minds. However, in the end, a good conversation is still the best way to get to know your user. A well-planned interview in addition to being a good listener is unbeatable and cannot be replaced when the goal is to establish a connection to comprehend how someone else is feeling. All these instruments help us to gather more information, evidence, and explore more results. However, they are only allies in our pursuit of understanding others.
Your user is a person who is out there in the world and probably wants to tell you how he feels when using your solution, you just need to be empathetic enough to listen to his call.
Thank you for reading until the end. Follow for more innovative ways to integrate psychology and design (and coding). I hope that this reading has provided you with valuable insights for your next projects. Until next time.
