Accessibility testing the payment journey
To get ready to open a pilot of the service in Scotland and Northern Ireland, we needed to understand how accessible the service was for people who use assistive technology. Doing this testing is one of the things that helps us to write an accessibility statement which is one of our legal requirements as a public sector organisation.
Assistive technology is software that disabled people use to help them access the internet, read long bits of text or write long paragraphs, as well as any other tasks they want or need to do on a computer. The type of assistive technology people will use varies depending on the “access need” they have. If someone has a mobility access need where they can’t use a mouse, they might use speech recognition which is like Siri. If someone has a visual access need where they can’t see the text on a website, they might use a magnification software to make everything bigger or they might use a screen reader to read out the content on the page depending on what they need.
To test the service was accessible, we recruited people who used different assistive technologies including screen readers, magnification and software designed for people with dyslexia. We tried to recruit people who used speech recognition, but on the day none of them would actually use their assistive technology for the service — this isn’t ideal and means we’ll be doing more accessibility testing a bit sooner than we had previous planned. This part of the service is very numbers heavy so we also included some people who don’t use assistive technology, but had severe dyscalulia or dyslexia.
Over 4 days, we ran 15 hour long remote sessions where we asked people to go through all of the pages on the payment journey, if we had time we also looked at some of the pages from the Project Enquiry Form. There are 11 core pages in the payment journey, but we also tested some of the error messages on these pages.
Overall, the service is pretty good for people using the assistive technologies we tested with (screen reader, magnification and read & write), but there are some bugs that mean it doesn’t behave in the way people expect.
Things that will make it harder for assistive technology users to access the service
Date fields
In the date fields, each field is set up as a “type=number” which meant that when screen reader users reach the field the screen reader says “date of spend, day, spin”. Spin tells people they can use the arrow keys to select the number they want. But because of how “type=number” fields work, this goes into negative numbers which is confusing for people.
They expected it to move them through a list of potential numbers that was limited to the potentially correct answers — so the day field would be limited to 31 and month to 12. The idea of arrowing to reach “2021” made people worried.
We could either limit the fields in that way, make them into a dropdown with a list of suitable dates or we could change the type to text so that arrowing doesn’t work. The standard practice for this pattern would be to make them text boxes so people need to write the numbers they need.
File upload designs
In the payment journey, there are two different patterns for the file upload which confused some people. It meant that people weren’t confident about what would happen once they’d selected a file.
These types of inconsistencies can undermine a users confidence and make them unable to complete their journey.
The file uploads don’t have an explict way to tell people what’s happened after they’ve picked a file and clicked the “add evidence” button. It adds it to the list below the button, but doesn’t announce this change. This means that users, particularly screen reader or magnification users don’t know when they’ve successfully uploaded a file without exploring the page.
In the current design, there isn’t functionality to remove a file that’s been uploaded, but if users choose and upload another file it will replace the exisiting one. This caused a lot of confusion for the people who took part in the research.
Some people thought choosing a second file, would add it to the list of uploaded files.
I don’t think it would delete that one, but add another one to the list, because you might not have just one receipt, so the option to add another one would be good.
Participant 10
Others looked for a delete button or tried to use common delete methods, like the backspace. Some people tried to click on the file to see if it gave them any options, but this then downloaded the file they’d uploaded.
Other people weren’t sure at all what to do and talked about closing the browser or panicking. They thought that once a file was uploaded it couldn’t be removed.
There doesn’t seem to be an option to remove it. It’s just there as evidence and I can’t remove it. If I’d uploaded it, I might panic and then close it and come back to see if it was still there.
Participant 10
Things that will make it harder for all users of the service
What is an action
It’s not clear for users if links are to see information or if they’re to do an action. Throughout the payment journey, we use links as both actions and as a way to access information. This makes it difficult for people to know what will happen when they click a link.
It also makes it more challenging to know the difference between a link and a button, particularly when both options will take the user to another page.
“It’s the blue links and green boxes — which ones are actions, which ones are just for information.”
Participant 11
We should be consistent with the elements we use throughout to help make users more confident using the service. As before, these inconsistencies can erode a users confidence and stop them from completing the task independently.
Visual layout
The layout of some pages make it hard for people to understand the content on them. The use of tables meant that it was harder for them to understand what the page was telling them or what they needed to do with the information. The amount of information was particularly difficult for people with dyslexia or screen reader users.
The page for reviewing a project spend caused a lot of anxiety. This layout is used twice in the payment journey, so it’s important that we get this page right.
“For me, already that’s jarring to me that table. I know it’s split up but there’s so many numbers I cant read that.”
Participant 5
Deleting an item of spend
The way that people can delete items of spend from their list is confusing. At the moment, deleting an item is a three step flow. People pick an item to delete, they confirm it’s the one they want to delete and then are shown the item that’s been deleted with a message that confirms it’s been deleted.
The third page surprised people, some people didn’t expect to see the information again after they’d clicked “confirm and delete”. One person said
“Wouldn’t expect to see that table again. I don’t need to see that it would just confuse me.”
Participant 5
Another person said
“Do not think it is necessary, confirmed it was being deleted in second step, does not think it is needed to have confirmation that it has been deleted another thing to look at!”
Participant 8
Other people thought that this was another confirmation and that there would be a way to come out of this page without deleting the item. One participant said
“Its almost like a double check safe, this the one you deleted, if you didn’t mean to delete it, you will have to put it back in”
Participant 13
Another said
“Page showing confirmation of deletion, ‘do I want to do it’ If go back ‘will go back and not delete’”
Participant 9
This is not the case in the current design. From the third screen, there’s no way for users to recover the item.
Error messages
The language in some error messages are unclear which makes them difficult to correct. We intentionally tested two error messages on the bank details page, one where the payment reference was too long and one where people were not allowed to use a space.
The error message for the second example is difficult to understand which makes it difficult for users to fix the problem and move to the next page.
Correcting the first error message, “Payment reference must not be greater than 18 characters”, was tricky for people as the field doesn’t tell them how many characters they’ve entered.
Most people corrected this error by counting the number of characters they’d used and then adjusting the reference. Other people guessed how long it could be so saw this error multiple times.
“The reference is too long. I’d physically count the characters. I’m sure there’s a short way but I’d just count them. I’d probably just take out National Lottery and leave Lottery Heritage Fund.”
Participant 6
The way that the errors work, it shows one error at a time for the field. This means if a user writes a payment reference that breaks multiple rules, they need to resolve them one at a time which adds to the frustration and makes it harder to move through the journey.
“You have to do trial and error, you only find out when you click submit and then oh there is a problem which means you keep having to go back and forth.”
Participant 4
The second error message on this page, “Payment reference must only contain numbers and letters”, was harder for people to correct. This error meant that people weren’t allowed to use spaces in the payment reference field, but the phrasing makes this difficult to understand.
Some people thought it meant they had to include numbers and letters which didn’t resolve the error when they tried it.
“Now it is saying ‘should only contain letters and numbers’ Gonna have to put a number there.”
Participant 4
Most people needed to be prompted to take the space out of the reference they had entered.
Another common error we saw was that in the number field on the Project Enquiry, users would add a comma in the number they were typing like “105,000”, but this wasn’t allowed in the validation so they got an error message.
This message still wasn’t clear, but more users understood how to fix the problem without help.
What happens next
We’ve already started work to fix some of these issues — particularly the date fields and the language of the error messages — but other issues that we’ve identified in this testing are bigger pieces of work. This is things like the way we do file uploads in the service and how we present budget information.
These design elements are key to this journey, but also other steps in the service so it’s really vital that we spend the time and energy getting these right and using them consistently throughout the service to help maintain user confidence and make sure the whole service is accessible.
It’s likely that we won’t pick up work on these design elements until after the pilot in Scotland and Northern Ireland.
