When UX and QA collide
When quality assurance analysts (QAA) are going through their testing scenarios UX problems can often be stumbling blocks to the quality assurance (QA) process. While UX should not be the focus of QA, if the experience does not make sense for the user, then it will get in the way of attaining a quality product. So how should UX issues be handled in the UX process?
“QA and UX go hand-in-hand. I often come across UX bugs when the actual response doesn’t match the expected response. While the system may be operating correctly, the user experience is not how it should work. These issues get brought up to the Product Owners for a discussion on if the design, user experience, or system can be improved.” — Katelyn Meyer QA Analyst at Ascension Press
Having a separate UX QA process could be helpful to finding UX bugs before they are found in the traditional QA process. This could take place at the prototype stage. Testing procedures per use case executed by the QAA or other team member, but preferably not the UX Designer as it often takes fresh eyes to spot the error, would help spot specific experience deficiency.
I have written once or twice (wink) about the importance of talking to and testing with actual humans. Speaking to users of the product will help with UX QA significantly as it can reveal experience gaps that may not be known. Asking users questions like, what did you expect to find here at this step can be very eye opening as it is a specific question that a user may not comment on without prompting. Here are some other questions that would be helpful during testing to identify UX bugs, errors or oversights.
- (On a particular screen) what do you expect this button to do?
- What step do you think is likely to come next? / What is your best guess?
- What menus are you surprised to see? Why?
- What menus did you expect to see, but don’t ?
- What documents/elements/functionality did you expect to see today that you didn’t?
Hopefully, you have spoken to your humans prior to the prototype stage during customer discovery so that you know the answer to most of these questions. The key here is to dig in and try to poke holes to uncover any oversights that you have missed during the design phase. It is helpful and unrealistic to think that the UX Designer will think through every eventuality of a procedure sitting at their desk not talking to users. Design — good design — requires observation and conversation. Intimate knowledge of the problem requires intimate contact with the humans that will or do use the product that attempts to solve the problem. Consider running a separate UX QA process (and no this doesn't’ need to be lengthy) but talking to users in user tests is essential and will up the UX QA quotient a ton