What I learned from IDF’s course on usability testing
One of the most crucial skills of a product designer is being able to empathize with the user and center the design process around them.
At Appear Here we’ve been steadily improving our usability testing sessions over time. However, I still wanted to expand my theoretical knowledge so I decided to take a course at the Interaction Design Foundation to learn more about usability testing’s best practices.
This blogpost is a summary of the main learnings I took from the IDF’s course and includes the slides of the ‘product talk’ I presented internally to the team. It touches upon good practices and explains what are the key differences between each type of usability test.
Topics include:
- Types of usability tests
- Problems with incorrect testing
- Usability test step by step:
1. Create a plan
2. Run the test
3. Analyse test & create a test report
At the end of this post, I also provide my review on the IDF course if you are thinking about doing it yourself.
Types of usability testing
Low-fidelity test — Could be either using cut-out paper or using an early-stage digital prototype (eg. using a low-fi design in InVision). Good for early-on feedback.
High-fidelity (fully functioning prototype) — A final test before releasing the product to test the experience as well as the components interface (information architecture, menus, navigation, branding, etc.). It can also be used to test an already released product to be continuously improved.
There is no best or worst type of testing, both these tests should be used at different stages of design/development.
Summative: Has a large sample size (50 users or more) and its goal is to capture metrics (eg. time spent on task). It uses a hi-fidelity design and it’s usually remote as it’s more cost-effective. Some known websites for remote testing: usertesting.com , teston, usertesting.io, Usercrowd.
Formative: Its purpose is more focused on analysing how a user behaves rather than capturing numeric data. It’s used for gathering quick insights from a smaller user pool of around 5 users. It uses a low-fi prototype in an informal testing setup.
Formal: Usually it’s set up in a lab, using sophisticated tools to record the session and aims to have a more controlled environment. However, they are expensive & time-consuming.
Informal: It consists of meeting a few users regularly in an informal setting. For that reason, it is less focused on metrics and more in qualitative testing. It is especially useful to test concepts early on.
‘Down the hallway testing’: Having a friend or colleague to test your design. It’s not advised for a reliable test for multiple reasons:
1. They are usually the wrong user target
2. They can have more context than a real user
3. They may be biased towards pleasing you
However, this type of testing can be used as a dry-run before you meet the real user.
Problems with incorrect testing
1. Create a test plan
Below you can find a Test Plan example sheet to prepare for a test.
- Task objective: what are you testing and why? Be precise here.
- Test task (the most important): what you’ll communicate to the user so they can start their task — usually a question.
It should use informal language easy to understand, and it should not lead to an answer (eg. don’t use the exact word used in a button to not guide them too much). A good idea is to always print each test task for the user to read himself. - Comments: things you want to check with your colleagues or any other notes relevant to the task.
- Needed: provide the link to where the prototypes are or the pdf that needs to be printed.
2. Facilitate test
Remember that most users have never done or heard about usability testing, so try to give a brief explanation of what they will be doing.
It’s important they know you want to test the product, not them — reiterate that there are no right or wrong answers. And also that you will not get offended with negative feedback — any feedback is good feedback :)
This is not essential for the usability test per se but could be useful to use the opportunity to ask them a few more questions:
Pre-test questionnaire: When they arrive and while they wait, ask them to fill in a quick profile so you can better understand your user. (Are they tech-savvy? what tools do they use on their job?)
Post-test questionnaire: This can be used to evaluate more opinion-based answers on your brand and style or to understand how the user felt about the task so you can get their perception of difficulty.
3. Analyse data & create a report
After the test, try to put down your findings by listening to the recording as soon as possible so you can share it with the wider team. Also, it’s useful to keep a written summary for the future so you don’t have to listen again to the recording.
This can be in any form of shared document (google doc, Notion, confluence page,…) and always attach the original recording to this summary so you don’t lose it.
These are some metrics you can use to measure your tests. The most common and easy to capture are: Number of errors and Success/failure rate.
You can make a graph out of this data to add to your report.
Obviously, these metrics are more useful if you’re testing a wider sample of users, not just one or two. But as I mentioned before, these are not essential for an informal usability test where you are more interested in capturing insights and testing early concepts.
Course review
My post is just a brief summary of the course, so if you are wondering if the IDF course might be of your interest, here’s my review. FYI: I’m in no way associated with the course, nor paid to review it:
- It gives a good idea of what is being done in big and smaller companies with more emphasis on professional usability testing. It can feel a bit corporate sometimes but I found it useful to know the standard practices so I can adapt it to a more realistic startup process. (I’ll try and write another post on how we conduct tests in an agile environment at Appear Here).
- The course provides useful materials you can use on your own usability tests and gives a certificate at the end.
- The membership gives you access to all courses on their platform for a year. I’m taking the “Psychology of Interaction Design: The Ultimate Guide” next.
Thanks for reading! 👏
If you are interested in the subject, check out my latest post ‘The ultimate Notion template to run efficient usability tests’
Looking for a space for your store? Book a space here
