Four Things We Learnt From Facilitating Usability Testing Sessions With Blind Users
Recently two of my colleagues (Todd Moore, Stephanie Moss) and I tested parts of SEEK’s site with vision impaired users.
It was my first time facilitating usability sessions with users who entirely rely on screen readers to navigate the web.
We learnt a lot of specific things about our site — things that worked and things we needed to improve.
However, there were four things we learned which were not necessarily about specific issues with our site — these were lessons which will be useful the next time we run usability sessions with vision impaired users.
Hopefully, you will find them useful too.
1. Just because someone is blind, doesn’t mean they are an expert at using a screen reader
I made the wrong assumption that anyone who is blind is going to be an expert screen reader user. How else will they get around the internet, right?
Wrong.
Just like there are varying degrees of computer literacy and proficiency in sighted people, there are the same variations with people who can’t see. This sounds, and is, so obvious. However, I assumed that being more dependant on technology, like screen readers, would force blind users to master the technology.
How it impacts usability test facilitation
When planning a usability session with blind users, allow more time. This is not to say that blind people are slower at navigating the web — the reality is that many websites are not optimised for navigation with screen readers. Mistakes happen, and those mistakes take longer to recover from when using a screen reader.
If you’re testing specific features you built for blind users, be prepared for them not to be discovered, and plan for how you will handle that in your session.
For example, we implemented ARIA Regions on our site, only to find that none of the participants we tested with used them.
2. Being blind may affect people’s propensity to learn new tools and technologies
Learning new tools and technologies can be hard (I recently gave up on trying to figure out how to use Snapchat — true story). Now imagine doing it without sight. It’s harder, right? It’s harder because most online tools and sites are not designed with blind people in mind.
One of the things we were keen to learn was what tools our participants used. We expected that they would switch between different screen readers, devices and browsers, depending on the task, location and context.
What we found was that they all stuck with a combination of tools they were familiar with. When questioned, they quoted high effort associated with learning new tools as a deterrent. This finding may differ depending on your user base and sample size.
How it impacts usability test facilitation
If you need to test different browsers, screen readers and devices, consider using a survey to pre-qualify your participants in detail.
3. Getting comfortable facilitating with blind users
To be honest, I was very nervous before running these sessions. I had no idea if the script I prepared was up to scratch. I found it really difficult to gauge upfront how long we would need to complete the tasks.
Most of all, I was worried I would say something insensitive, something that may offend the participants. I tried hard not to use phrases like ‘tell me what you see’ or ‘tell me what you are looking for right now?’. Instead, I used phrases like ‘tell me what’s happening now’ or ‘tell me what you’re searching for’.
It turned out I was stressing for no reason. The participants regularly used phrases like ‘I’m looking for…’ or ‘I can see that x is not working…’
How it impacts usability test facilitation
My advice would be: just relax. Really listen to your participants, hear how they speak and echo the language they are using.
4. Blind users are amazing at creating mental models of websites
One of the things that really struck us is to what level of detail our participants described the structure and the content of the pages they visited — they formed detailed mental models of the pages they interacted with.
The descriptions didn’t always match how the site was actually laid out, but that made them even more interesting — they gave us an insight into how our site was perceived by blind users.
How it impacts usability test facilitation
This is an opportunity to learn about how you should structure the content of your site and pages for blind users. Next time we run these sessions, I will definitely include an activity where we get participants to describe the structure of the pages they are interacting with.
It’s important to note that mental models of sighted users often don’t match the system model either — this is where we have a gulf of evaluation. Understanding and reducing the size of the gulf of evaluation helps us design better experiences, not just for screen reader users, but for all users.
Aside
This reminds me of the amazing artwork by Alby Letoy who was commissioned by TravelSupermarket.com to produce illustrations of travel destinations, based on descriptions provided by visually impaired people who visited them.
Here is one of those illustrations
Like with anything, practice makes perfect. I’m looking forward to applying these lesssons the next time we run usability testing sessions with blind users.
Big thanks to Kayla Heffernan for a detailed review of this post.
