Sometimes I hear the phrase “Users lie” from some colleagues, usually it comes from people who are sure that UX research is something like a unicorn, everyone talks about that, but no one has seen it in reality.
Users say one thing, but behave in a completely different way. I know I saw…
In fact, everything is not so simple here. Let’s figure it out. Can users lie? Of course they can. Like all people on the earth. Why does it seems sometimes users lie?
Here are a few reasons.
1. The user does not remember something.
Most often, researchers invite users with specific experience for testing.
For example, “It is necessary to take active users who visited the service at least 3 times in the last month”. That is, we know very well that the user we invited was on the service at least 3 times in the last month. However, there are situations when the question “when was the last time you visited the service?”, We get the answer “oh, I haven’t logged in for a long time, probably 2–3 months ago”. Lying? Or doesn’t remember?
Our memory is not perfect, one of the reasons may be that a person does not remember what he did a month ago, especially if there are inactive users in our sample.
Do you remember what you had for lunch a week ago, hmm?
2. Several people work from 1 account / 1 person works from several accounts.
Yes, it happens.
Most often, this is an attempt to save money. For example, there is one subscription and everyone uses it, there is no need to buy multiple subscriptions. Or there is a limited set of tools for an account, for example, in miro there is a limit on the number of boards, so many create multiple accounts.
That is, according to analytics, you are sure that the user visits the service every day, but he swears to you that it was a very long time ago.
Ask if anyone else is using the current account. You can also cheat and ask if someone from the family uses this service, and then clarify whether they have their own personal accounts or all from one;) The respondent may be afraid to tell you the truth, so that it does not happen, try placing him at the beginning — you are a friend, not an investigator.
3. Talk about the future, the problems of researchers.
Newcomers to UX research periodically make one big mistake: asking questions about the future. The problem is not that such questions exist, but that they do not know how to work with them.
For example, you can ask the question:
Moderator: Will you buy our product when it runs?
Respondent: Of course! Great product!
Finish with this and with the widest smile go to refine the product, because you have just been told that they will buy it. And then you can scale these results to all users before going to bed. Everyone will buy! Then it will be a little sad that the product never took off, a lot of resources were spent, and all users are liars, UX research is bullshit!
Now imagine a user who came to you for a test, you ask him a question:
Moderator: Will you buy our product when it runs?
Respondent: Of course! Great product!
Moderator: Have you ever used similar services before?
Respondent: No…
Then you can find out the reasons for what he used and uses them, and so on. The bottom line is that in fact, the respondent has no need for our product. We need to dig deeper: maybe we didn’t select the right sample, maybe our product does not meet the needs. But the users did not lie, they said what could happen, but this is not accurate …
4. We are not perfect.
Of course, if you drop a coin on the floor, then with 100% certainty it will end up on the floor. But when we work with people, it doesn’t work that way. There are too many factors that can affect the result.
For example, I want to appear better than I really am;)
Moderator: Do you play sports?
Respondent: Of course!
Moderator: How often do you go in for sports?
Respondent: Every day! (no, no, nothing like that…)
In such cases, you need to ask a couple more questions so that you can understand how close it is to the truth. What sport do you do? Why did you choose this kind of sports? Do you use any apps? Which? Why? (based on the type of sport, we ask more specific questions). Do you somehow measure the results? When was the last time you did it?
After such questions, you will understand whether they are lying to you or not. There are exceptional liars, but that’s how it turns out. You can double-check the analytics, if they frankly lie to you on several points (and you know the whole truth from the analytics), then it’s better to finish the test. Nothing good will come of it…
In conclusion
Do not pretend that UX research is something strange and has nothing to do with analytics. On the contrary, the more you immerse yourself in research, the better you understand how it works, how it can help the product.
All people lie. It’s worth recognizing. We just have to find methods of how to work with it;)
