Dipping Your Toes First: A Guide to Usability Testing
What is Usability Testing?
You’ve finished building your app and you think to yourself, eh, looks good enough. But perhaps, it isn’t.
As designers and developers, you’ve seen your system 10 too many times. You’ve understood the ins and outs of your application. You are simply biased. It’s time to put your product in the hands of real users and see how they manage.
Usability Testing is like a test drive. We let real users fiddle with our apps and if afterwards we identify a problem, then back to the drawing board it is.
Advantages of Usability Testing
- Catch problems before they go public: once you’ve released your product, it’s going to be a whole lot harder to contain the problem. It’s tiring and it’s costly.
- Boost customer satisfaction: happy users are return users. If we fix the problems, then users are going to have better UX.
- Gain insights to how users actually use your product: watching real users use your product can give you real insights, which you can then use to enhance your product even more.
Methods of Usability Testing
Quantitative vs Qualitative
Qualitative testing is all about the ‘why’ behind user actions. It’s less about numbers and more about understanding user behavior, motivations, and feelings. You’re not counting how many people clicked the wrong button; instead, you’re asking why they clicked it. This method often involves interviews, open-ended questions, and observing body language.
Quantitative testing, on the other hand, loves numbers. It’s about measuring data: how many users completed a task, how long it took, or what percentage of testers encountered a specific issue. This data is great for making graphs and charts that show you exactly where the problems lie in the usability of your product.
Moderated vs Unmoderated
Moderated testing means there’s a facilitator guiding the session. This person can observe, ask questions, and clarify tasks as needed.
Unmoderated testing is more hands-off. You set up the tasks, provide the tools, and then let users go at it on their own. This can be less intimidating for participants and more scalable for you. It often gives a truer picture of how the product will be used in the wild, as there’s no one to influence the user’s behavior.
Remote vs In-person
Remote testing is done, well, remotely. Users test the product from wherever they are, which can lead to more organic interactions with the product
In-Person testing involves face-to-face interactions, typically in a lab setting. This can be great because you can control the environment and equipment, ensuring consistency. It also allows for more detailed observation and immediate follow-up questions based on user reactions.
Steps of Usability Testing
- Create a prototype: Depending on what you’re testing for, it doesn’t have to be polished. Sometimes, just functional is enough. This could be anything from a series of wireframes to a clickable prototype. The key is that it must represent enough of your product’s functionality to evoke useful feedback from users on its usability.
- Come up with a test plan: This plan should outline what specifically you’re testing, the goals of the test, the tasks you’ll ask users to complete, and any specific questions or metrics you want to focus on.
- Recruit target users: The recruits should match your target demographic in terms of age, familiarity with tech, professional background, etc.
- Find a suitable location: For in-person sessions, a quiet, controlled environment free from distractions is ideal. For remote tests, you’ll need to ensure participants have the right tools and technology setup beforehand.
- Run the test: Execute the test plan and observe carefully.
- Document the test results: As the tests are running, document everything. Capture video and audio if possible. Good documentation is key to understanding the nuances of user interactions and the issues they encounter.
- Analyze results to find insights: Look for patterns or recurring issues. Analyze both qualitative feedback and quantitative data to identify what’s working and what’s not. This analysis will form the basis of your recommendations for improving the product
Desktop vs Mobile
Testing for websites and mobile apps isn’t quite the same. Aside from the obvious size difference, users also behave differently on desktop and mobile. Here are some key considerations to keep in mind when planning usability tests for each platform:
- Interaction methods: Desktop users typically interact with a mouse and keyboard, offering precise control, whereas mobile users rely on touch. This affects how interfaces should be designed and tested. For instance, clickable elements on mobile need to be larger to accommodate finger taps.
- Screen size: Desktops have more screen space, allowing for more content and complex layouts. Mobile devices, with their limited space, require simplified designs that prioritize essential content and functionalities.
- Context of use: Mobile devices are often used on the go, so mobile apps need to be efficient and straightforward. Desktops, generally used in more stable environments like offices or homes, can handle more complex tasks and multitasking.
- Battery life constraints: Mobile usability testing should also consider the impact of app usage on battery life. Efficient battery usage is crucial for mobile, whereas it’s generally less of a concern for desktop applications.
- OS and browser differences: Do test across different combinations of OS and browsers and adjust accordingly.
[UPDATED] Example of Usability Testing: Before and After
vlecture already has a working prototype, we can now perform usability testing.
We designed a test plan, which template can be viewed here
From one respondent we acquired the following results:
After receiving the results, some problems were brought to our attention.
To illustrate, the following is how vlecture’s page initially looked like.
We analyzed the issues reported to determine what we should improve on
- Issue 1: The layout feels cluttered
Usability heuristics violated: User control and freedom.
Cause: Users feel a lack of control over navigating the app and finding what they need due to the cluttered content. The frame around the note sections and the study panel coming in from the right takes up almost half the screen, even though the note is the main feature on this page.
Fix: Simplify the layout by reducing unnecessary elements, grouping related items together, and ensuring clear visual hierarchy.
- Issue 2: The colour choices are visually overwhelming
Usability heuristics violated: Aesthetic and Minimalist Design.
Cause: Visually striking colours are used in an excessive manner, even though they are supposed to be used to draw attention. This makes the user confused on what to focus on.
Fix: Use a more minimalistic color palette, ensure sufficient color contrast for readability, and use color consistently to convey meaning.
- The note generation feels slow
Usability heuristics violated: Flexibility and Efficiency of Use.
Cause: Backend takes too long to respond with generated note, making the service feel slow and inefficient on the user.
Fix: Evaluate the backend infrastructure to ensure it can handle the load, consider scaling up server resources or enforcing quota as constraint if scaling up is not possible.
After doing the analysis and implementing the fixes, this is what the redesigned page looks like.
Here are what changed:
- For issue 1: We made the sidebar collapsible so the majority of the screen can be utilized for notetaking. The Study Panel only consisted of two buttons so we factored them out as thin buttons to the side of the note sections.
- For issue 2: We redesigned the page to be mostly neutrals (black, white, gray) and reserved purple for the two buttons as to draw the user’s attention to those signature features.
- For issue 3: We limited the note generation to 5 uses a month so as not to overburden the transcription service.
