Testing product copy with the right questions
It’s tempting to ask ‘did you understand that?’ when you’re testing copy with someone. But it’s unlikely to give you really useful insights. When giving their answer, people aren’t trying to mislead you. It’s just that it’s entirely possible to understand every word on a screen and misunderstand the overall meaning. In this post, I’ll share some tips and tricks to get the most out of product copy testing.
Before we get going, it might be good to explain what we mean by ‘product copy’. We’re talking about any written content on our website or App — from microcopy to help explanations, and all the little bits in between.
To find out if our users really understand our product copy, we ask lots of different questions. And we always test in context with a prototype, where visual design and copy are together.
We’re especially interested in understanding:
- If the language used is familiar to our members
- If our copy works with how someone scans the screen
- If people understand the key messages
Familiarity
At Bulb, we speak in a normal, non-corporate voice — the voice of a friend. So we want to use language that is familiar to our members.
To find out what’s most familiar, you can do something a little bit odd. Try starting each session with a discussion about something that isn’t actually your area of interest. Why would you do this? Well, if you dive straight into the topic, you just prime people with the language you’ve used in your questions. People end up talking back to you in your own internal terms.
Example:
We weren’t sure how people referred to their prepayment meter. Was it a ‘prepay meter’, a ‘top up meter’, a ‘prepayment meter’ or something else entirely?
We needed to find out for both product copy and communications. We started the session with a general conversation about using appliances at home, and gently guided them towards meter chat.
We learnt ‘prepayment meter’ was most commonly used. People say they ‘top up’ their meter. We’ve been using both phrases ever since.
Top tips:
- Guide people gently towards topics without using key terminology
- Use broad questions (e.g. how does that work?)
- Note down a mini glossary of the terms people naturally use
Scanning
How can you uncover how people scan a screen? Why does this matter?
It helps you to answer lots of questions. What copy draws their attention? What do they skim read and barely mention? How are they interpreting icons or symbols alongside text? You can use findings to ensure your copy is working cohesively with your visual design, and make changes if it isn’t.
Example:
We redesigned our payments & statements screen. The page includes lots of information: balance, payments, credits, energy usage and more.
We asked users to think aloud as they navigated the page. We found that people focused on reading ingoings and outgoings from the bottom of the screen to the top, concentrating on the right hand column. They were doing this to make sense of the balance towards the top of the screen.
This insight helped us figure out where to place the call-to-action to top up a low balance. It’s now on the right hand side of the page as this fits most naturally with how users scan.
Top tips:
- Give people a prototype, a scenario and a task to complete
- Ask them to think aloud as they navigate through the prototype
- Reassure them that nothing they say will be right or wrong and that they’re free to say whatever they can think of
- Note down the order of information they relay back to you
Understanding
When getting people to think aloud, you can also watch out for anything people have to re-read or are unable to put into their own words. That’s a red flag that you need to rewrite copy to make it clearer.
Example:
Bulb recently became a Feed-in-Tariff supplier. We designed an online sign-up form for members to enrol in the scheme.
The form included a lot of recommended regulatory language. In usability testing sessions, users navigated to the end of the form and successfully completed the task of signing up.
However, they didn’t understand all of the information presented. They weren’t able to explain our explanatory copy back to us.
We explored why they were confused (e.g. was it unfamiliar terminology?) and iterated the copy based on these insights.
Top tips:
- Ask people ‘can you explain that back to me?’. This questions helps you to measure understanding
- Ask ‘why do you think that?’ If they’ve misunderstood, this question helps you to understand what might be confusing. You can immediately work on improving that copy and evaluate in future testing sessions. If they do understand, it helps you identify what’s worked as a signpost or explanation on the page. You can make sure to keep that consistent in future iterations
