Stop Asking Users What They Expect To See

“What are you expecting to see at this point?”

“What are you expecting to see?” is one of the most misused questions in user testing.

I can’t count the number of teams I’ve seen switch to this question when their user gets stuck in their prototype. Almost everyone is taught to ask it. However, what we aren’t taught, is WHEN to ask it:

“What are you expecting to see [at this point]?” is a UI/UX question. The PURPOSE of this question is to understand the user’s intuition for the flow of the product, so that the design can be made intuitive, elegant, and subconsciously aligned.

It’s a fantastic and powerful question…. ONLY when you are already confident what the core value proposition for the user is. Once you know that, you are solving for how to most effectively deliver the value proposition — that’s the right time for this type of experiential question.

However, if you ask it in early design, before you know your value prop, while you’re still less than about 65% confident about your solution, it will immediately derail and stall the test.

The reason for this is:

If your user gets stuck on a product flow with a validated value proposition, it’s because they’re CONFUSED. The FLOW isn’t work for them, so you can ask them what they really expect next in the flow.

BUT, if your user gets stuck on a product BEFORE the value prop is validated, it’s because the value prop ITSELF didn’t land.

In the first case, they aren’t receiving value because it’s blocked by the flow. In the second case, they aren’t receiving value because it doesn’t exist.

If the user stalls on a value proposition -level test, this means that the moment we ask, “what do you expect?”, they actually sort of internally panic — because we just asked them to invent our value prop and they have no idea either. We accidentally just hit them with the full emotional discomfort of the unsolved design.

For all the times that a team we were working with saw their test fail and fell back to the familiar “what do you expect?”, I have not seen a single user come up with a useful answer.

Instead, one of two things happen:

1) The user flails, feels uncomfortable, feels like they’re failing you, and emotionally disengages from the test. They say something like “I’m not sure…” and then there’s a bit of awkwardness to iron out.

2) The user tries to be helpful and answers at a UX level, the experimenters then take that as an answer, and the conversation switches into the realm of UX instead of value prop. Before anyone knows it, the next 10 mins of conversation are about the design team and user working together to find the best way to convey a value proposition that the user doesn’t actually want.

In both cases, the overall tenor of the conversation leaves the realm of the impactful/innovative, and shifts into a dissatisfying slog through low-impact, existing possibilities

Now, this is not particularly big deal. It’s easily recovered with a quick bout of active facilitation and an energy reset. As long as we actually DO the energy reset, we can pick up where we left off.

The 1–10 Rating System

Here is the question that I like to ask instead when a user gets stuck at the value prop level:

1) On a scale of 1–10 (1 = horrible, 10 = mind blowing, 5 = fine/as expected), how valuable is this solution to you?

2) What would make it a 9 or 10?

You are actually, at this point, STILL asking the user to unstick you and invent the value proposition.

However, now you’re not asking them to solve your whole design in one single step: you’re just inviting them to talk about themselves at a high level, informed by the context already created by the prototype. There is no idea generation or solutioning on the part of the user — just a reflection on their own sense of value, inspired by your prototype.

“What would make it a 9 or 10” unleashes a fountain of useful information.

Additional tips for this technique:

a — Make sure to ask the baseline 1–10 on the existing design first.

b — Make them pick a number. Many people are reluctant to pick, they’ll say, “oh it’s fine”. It may feel awkward to push, but get it done, it’s worth it.

c — Make sure you EXPLAIN the scale. (1 = horrible, 10 = mind blowing, 5 = fine / as expected.) Otherwise, depending on what country you’re in, people will assume anything from a 7 to a 9 means “acceptable”, and you won’t get enough nuance to actually express the type of impact you’re looking for.

Lastly, it’s important to realize you’re not always looking for a 9 or 10. In fact, on this metric, the vast majority of the time, your final solution will be lower.

The important thing is to find a value proposition solution that reliably achieves at least a THREE POINT IMPROVEMENT to the status quo.

In order to find that, we ask a third question in between: “On the same scale of 1–10, how would you rate your current experience with X?”