The Powerful Fix to the Usability Testing Process

We know usability testing is important and there are a ton of resources on how to organize and facilitate user testing, but how do we format it and present our findings to non-UX designers? How do we make it more efficient with a quicker turnaround time?

I ❤️ doing user testing and have completed over 50 usability tests and on the way developed a quicker, easier and more beautiful way to track and provide user testing results.

This uses the standard usability lab testing. If you are new to this concept I recommend reading Don’t Make Me Think and use the below in your process.

Not interested in doing user testing yourself? Get in touch and maybe I can help.

The problem with the standard usability testing process

When I first started conducting structured usability testing I was the solo designer and very junior. I had finished my beautiful InVision prototypes and was excited to start conducting user testing for the first time. There wasn’t much documented online on how to format the information during the user test and display the results. So, I printed out one copy (of 6 sheets) for each of the 5 individual users being tested. I would start the test by having the user go through each step and would scribble my thoughts on the printed document. Afterwards, I would grab my bundle of messy printed sheets and type my findings into a separate text document for each user with their name as the title. I would take each of the typed documents and put them side by side to compare them to try and find what issue kept coming up most often. It would take me a long time to find the biggest pain point to address first. I would then provide a wordy report to the founder and developers who skimmed through it and when they asked for references on how I got to my conclusion I pointed to the ugly pile of paper on my desk.

Omfg overwhelming amount of papers and RIP trees.

The problem with receiving user test results

I once worked on a project for a charity app. There were UX Researchers already working on the project that provided me with the results of the first round of user testing. They gave me a separate typed 4-page document for each of the 5 testers that listed all the issues but not which were the most important. I had to once again compare each document for each of the users side by side to prioritize the issues based on how many users had trouble with the same thing. I would then have to refer back to the taped video to understand what part of the app they were having troubles with and what step they were on when they had problems.

Old way — Here’s an example of comparing three of the tests.

After I did my analysis I provided the proposed design changes to the client, my boss and developers. When asked how I came to these conclusions, I could explain my reasoning but wasn’t able to display the data in a simple format to back up my informed decisions.

These processes were time-consuming, disorganized, unsustainable and had a slow turnaround time.

The new beautiful process 🌟

The issue at hand was there wasn’t a way to easily visualize the results. quickly learned humans process visual data better, but we were currently only providing a bunch of text.

I developed this way of displaying visually in a comparison where there were issues and where things worked. It keeps everything in one file instead of printing 5 separate 5-page documents, typing the 5 documents, creating a report, then crying because it has become a mess. All the answers are in one nice sheet that you can easily compare each issue by looking at the column.

Here is an example of a couple screens that were tested with an example of one section of the test that I did for RunGo, a running app for travellers:

RunGo screens that were used for the user testing examples below.

User testing process sheet example, with no sorting yet.

Preparing the test

Here is a free Google Sheets template and example for you to use 😉.

1. Write your user interview questions in the first few columns (ie. name, technical skills, how often they run, questions applicable to your test).
2. After the questions type the steps/script to use read out loud when conducting the test in new columns.
3. After steps/script include final question in columns if needed.
4. Include a few columns at the end for notes.
5. Create a new row for the screen name above these questions and steps .
6. Create a new row for each user’s name.
7. Add conditional formatting for the beautiful colours: For “1” do green, “2” do yellow, “3” do red. Or whatever colours make you happy.

Conducting the test

1. As a standard usability test, have the user go through each task that you have already listed at the top of the sheet.
2. As the user takes the step type at the beginning of the cell:
   - “1” if they got it easily.
   - “2” if they had issues and eventually got it or there were other insights involved.
   - “3” if they couldn’t achieve it without help or there was a significant struggle.
3. Include notes in the cell after the number.
4. Include anything that comes up in “other notes” in the last column(s).
5. Directly after each user test schedule time to go over what you’ve written to make things coherent. Usually you’re typing at a fast pace and there will be ugly typos that you might forget what it meant later. Add additional notes where needed.
6. You can create a new sheet and sort it by number to see what has the most red to address first OR you can add up the numbers (optional).
7. Your test is now complete! you can continue to do your analysis to assess changes and can easily see how to prioritize each of these issues. Start with the most 3’s/reds, make suggested changes and work your design magic! ✨
8. Afterwards, when requested (by your boss, developers, clients etc.), you can provide the sheet as reference to your design decisions in a visual format. Although a big sheet, it was easy to compare their different responses and visually see where there were issues instead of a bunch of separate papers.

Real Life Example

Below I have sorted the results from the test above for the RunGo app. You can easily see now that it is colour coded. You can now easily see we discovered an issue (yay!) for the user to find the elevation graph. We could also see that finding a route nearby at 7km was easy for the user and doesn’t require anything to be changed.

The example but sorted to see that there was an issue with elevation

Conclusion and benefits of the new process

• I’ve saved hours and hours of work! At least 6 hours per round of user tests.
• Turnaround time for providing UX results is quicker.
• Everything is consolidated in one place, no more seperate docs.
• More sustainable. printed sheets and death to trees.
• Non-UX Designers (ie founders, developers, visual designers) are easily able to visually see where I got my conclusions from.
• Easier and more fun to read than a wordy report.
• WAY less stress while conducting the test.
• More accurate in prioritization.

So try it out! This is just my personal way and suggestion, let me know if you have any variations of this.

This is my first article on Medium, and probably the first article since highschool, if you enjoyed it please make sure to 💚 it below! 😘

Thank you to Nikkita and Kiyomi for feedback.

Want to hire me, work together, have questions or want to have a beer? Or are you using this in your class? I’d love to hear from you!
Email me,
visit my portfolio jayceeday.com,
or follow me on Twitter

💘

Subscribe to The UX Blog