Conducting usability testing with blind users- research facilitation tips

Image for post
Image for post
Photo by rawpixel on Unsplash

“By failing to prepare, you are preparing to fail “

— Benjamin Franklin

Before facilitating my first research sessions with blind users at Vision Australia, I prepared. In this post I’m going to outline what I did, and dive into some pragmatic tips on running these sessions so you too can be prepared.

First, I performed a literature review to understand the online and search behaviours of blind and vision impaired users, and to quell some common myths in our delivery teams such as:

  • “Blind users know how to use screen readers” — False. Not all do, just like not all sighted users are tech power users.
  • Blind users would just tab backwards” to excuse us not developing a usable tab order or correctly implementing ARIA interrupts. Also false.

As designers we know that assumptions are dangerous things, and that if you don’t talk to, and observe, actual users interacting with your product, you can’t design something that actually works for them.

Vedran Arnautovic forged the path ahead of me by running SEEK’s first usability testing with blind users, and he shared four lessons from facilitating usability testing sessions with blind users. These were:

1. Just because someone is blind, doesn’t mean they are an expert at using a screen reader

2. Being blind may affect people’s propensity to learn new tools and technologies

3. Get comfortable facilitating with blind users

4. Blind users are amazing at creating mental models of websites

As with everything, practice makes perfect, and while I’ve conducted hundreds of sessions these were my first four with blind users. Armed with the above preparations, I felt pretty confident going into them and learned some lessons along the way to improve next time. Here’s what I learned…

We wanted the participants to use devices in their natural environment. As these users were all fully blind, and they had no need for a screen, we found that the desktop was not connected to a working screen (which of course they didn’t know). We wasted a while in the first session getting one up and running for us to observe and understand where they were on the screen.

At SEEK we have printed consent forms that we get participants to read and sign before beginning any research. I overlooked preparing an accessible form, i.e. screen reader readable and using an online signature, so I awkwardly read the consent form aloud and asked for verbal consent rather than making them try sign paper. PDF’s can quite easily be made accessible if you put in the small effort, or it could even be a webpage.

At the time, we provided incentives in the form of VISA gift cards with printed instructions. Again, these were not accessible to the participants meaning they would have to ask for someone’s assistance to know the PIN and use the card, which isn’t very secure! We’ve now moved to digital gift cards which are, hopefully, more accessible (their website is a little sketchy on the details).

Limit interruptions where possible. Of course, sometimes probing is essential clarify actions or intentions, but if you speak over the voice over of the screenreader it can take a while for participants to get back into their flow. Try note down questions and leave them to the end.

We encouraged the participants to use the website as they usually would, which meant listening to the screen reader at an incredibly fast pace. Even if you listen to your Podcasts at 2x speed, it’s likely going to be too fast for your untrained brain. Stop trying and just observe and take notes on what they are doing with the keyboard. Ask the probing questions later. It would be great to use a tool that visually indicates the keystrokes entered as it is hard to follow what users are doing with the keyboard (because they move so fast).

I had two notetakers, and took notes myself as the facilitator, to keep up with the fast keystrokes and note where they appeared to struggle. Blind users are used to having to battle their way through badly designed websites so they may not mention usability issues they are facing. Just because users can get through the experience doesn’t make it usable, and we should be aiming higher. I also found it easier to note down things myself while facilitating than I do in sessions with sighted users, as I didn’t have to worry about maintaining eye contact.

Blind users generally take longer than sighted users to complete tasks, due to the way they have to navigate the website. We used a script we’d previously used with sighted users, where the largest tasks were at the end. Consider re-ordering for blind users.

I was a bit anxious about using words or phrases that would be offensive, or micro-aggressions, by asking users things like “what are you looking for on this page”? Visual metaphors for actions were common in the vernacular of the participants, and once they used one themselves I felt more comfortable echoing their language. e.g. “Click”, “scroll”, “see”, “look”. We did both have a laugh when I asked if they ever print webpages 🤦🏻‍♀️

I’ve also used the term ‘blind’ throughout this piece, echoing the language of these participants (they were all legally, and completely, blind not vision impaired).

Written by

UX Design Lead. Passionate about solving ambiguous problems with accessible solutions. PhD’ing part time.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store