UX KPIs and how to measure them
Do you know how satisfied users are with your product? And where do you get this information from? I think the answers can vary a lot, because someone is in constant communication with users and someone doesn’t have access to them, this is the reality of design today.
Knowing or hearing the big picture is one thing. Whether you can measure the UX, because for some it may seem abstract, the answer is yes, you can and should. There are a number of valuable indicators that can help you understand how good your design is.
Now we’re going to try to break down 4 metrics that help me in my work to understand how good my design is. These are so-called UX KPIs, quite a buzzword these days, but let’s not be skeptical, because we live in the era of big data, and they are collected by different companies for different purposes. Our task is to collect data and understand to what extent and how our product satisfies users.
What are UX KPIs?
KPIs are the measure of success in a certain activity or in achieving certain goals. We can say that the KPI is a quantifiable indicator of the results actually achieved. The performance indicators are usually different for each project and should always be determined individually. UX KPIs are some of the most difficult to understand and work through because we need to translate users’ feelings (and sometimes thoughts) into numbers to understand where we are going.
For a better understanding and measurement of UX, KPIs are divided into two global groups: behavioral and attitudinal.
Behavioral (What users do) vs Attitudinal (What users think)
As you may have understood, in order to measure different behaviors, we need to distinguish between user behavior and user thoughts. Why is this important to do? It’s simple: users can’t always tell you to their face what they’re thinking about, and their behavior doesn’t always reflect their thoughts. Human psychology is complex to understand and we won’t get into that now. Just know that to be sure about something, look at what a person is doing and try to find out what they are thinking or vice versa. This will help you compare the data and get an approximately correct answer.
Behavioural UX KPIs express in numbers what a user is effectively doing and how they interact with a product or website:
– Task success rate
– Search vs navigation
– User error rate
Attitudinal UX KPIs measure how users feel or what they say before, during or after using a product:
– System Usability Scale (SUS)
Task Success Rate (TSR)
Probably the most common and easy metric — TSR, the percentage of users who were able to complete a task. The task must have an end goal, so that it is possible to determine whether the user has coped with the task or not.
Let's imagine that you're testing the registration flow and the metric for success will be getting to the user's home screen. You have 10 respondents, 7 of whom coped with the task, successfully created an account and got to the home screen, and 3 of whom failed:
– User 1 could not find the signup form
– User 2 refused to enter their registration information
– User 3 forgot their mail info and couldn't pass the verification process
So the TSR will be calculated as 7 (number of respondents who passed)/10 (total number of attempts) = 0.7 x 100 = 70%.
Although the TSR does not say anything about why a user is not successfully completing a task, it is an initial, very valuable indicator. Therefore, to determine the cause of the failure, you will need to do further research or ask clarifying questions if possible.
Search vs navigation
The two primary navigation methods (navigating and searching) and how users interact with your product (through the search field or using navigation) are questioned. In many cases, the less the search function is used, the better the customer experience is.
You give a test task to some people to perform a certain task and observe how many people used search and how many people navigated to reach the goal.
Let’s say you have 10 respondents and 7 of them used navigation and 3 used search to accomplish the task. Then the percentage of people using navigation would be 7/10 = 0.7 x 100 = 70%, and search 3/10 = 0.3 x 100 = 30%.
User error rate
The user error rate (UER) is the number of times a user makes a wrong action that doesn’t lead to success.
Let’s take five respondents and the task they have to complete. For each respondent, we measure the number of errors or critical difficulties they encountered and determine that for this task, the maximum number of errors is 5 (scenarios):
User 1 = 2 errors
User 2 = 0 errors
User 3 = 5 errors
User 4 = 2 errors
User 5 = 1 error
Then we count the error rate. It will be equal to:(2+0+5+2+1)/5x5=0.4x100=40%
System Usability Scale (SUS)
According to its inventor John Brooke, the System Usability Scale (SUS) is a ‘quick and dirty’ tool with which you can test the usability of a product. The scale consists of a 10-point questionnaire with five possible answers each, ranging from strongly agree to strongly disagree.
Looking at the respondent’s answers and the corresponding number score for each response, you can tabulate the overall SUS score by using the following framework:
– Add up the total score for all odd-numbered questions, then subtract 5 from the total to get (X)
– Add up the total score for all even-numbered questions, then subtract that total from 25 to get (Y)
– Add up the total score of the new values (X+Y) and multiply by 2.5
Example scoring:
Odd = (4+5+3+4+3) = 19–5 = 14
Even = (2+1+3+1+1) = 25–8 = 17
SUS Score: (14+17) x 2.5 = 77.5
Odd — questions 1, 3, 5, 7, and 9. Even — questions 2, 4, 6, 8, and 10
Results table you can find here:
Summary
Using these metrics, you can already qualitatively understand user behavior. With a combination of usability testing, you can get a great picture. Don’t be afraid to combine different metrics and testing methods, but always know the context of which one will work in a particular situation. The metrics I have described today are not all possible ways to get the necessary information, but in my work I mostly use them. Good luck in practice and don’t hesitate to share your experience using and measuring different UX metrics.
