What I learnt from my first in-house usability study
7 tips you need to know before you begin
Gaining user insights is crucial to every design process. If you don’t know how your user will interact and behave with your product, how do you know it will be usable? Yes you can use your gut or your years of experience but there will always be a chance you might miss something; because most of the time, you are not your user, so as much as you try, you won’t be able to predict how they will use it. This is why user testing is crucial to our design process.
Always recruit more people then you actually need.
For this usability test the aim was to use 5 people. Many studies show 5 users highlight 85% of all usability issues.
I did however recruit 6 people — I thought I’ll add another 1 to be on the safe side. But turns out you should add another 2 or 3 to be on the safe side. Our first drop out couldn’t get out of work due to an emergency and our second drop out just decided not turn up… it happens. In the end we used a member of the reception team and he was more than happy to take an Amazon voucher off our hands for taking part!
Another tip: Amazon vouchers really incentivise people to turn up and actually take part so always good to offer one (we offered £25, but depends on how big your budget is).
Conduct a pilot study before hand.
Dress rehearsals are super important. And you need to make sure you do them properly. This includes getting people to go through all the paper work (such as consent forms, NDAs), explaining the study to the users, going through the full introduction. Even if you feel silly doing this with your team member — it’s important for timings and provides you with an opportunity to rehearse. You also need to test your equipment during this study, this means actually filming and audio recording the session so you can ensure everything is working. The last thing you want when it comes to uploading all the recordings at the end of the day to find the audio didn’t record for whatever reason (no, thankfully this didn’t happen to us — but it could!).
Plan your study well.
Ensure you’ve covered everything before bringing people in. You can start by completing the Usability Test Plan Dashboard.
If this is one of the first studies you’ve done, it will help to organise your thoughts and get the main pieces of information on paper. Print the dashboard onto A3 and fill in the sections. It will allow you to think through all areas such as your objectives for the test, the business case, the tasks you want your users to go through. You can then get another member of the team to review this to ensure you’ve thought about everything.
Create a consent form.
Check your company process beforehand. Do they have a procedure for discussing projects with users or with involving users in projects? They may have an NDA document in place which they use (really depends on the project if this is necessary). In my scenario this wasn’t relevant as we weren’t revealing confidential information or designs. However, I did create a consent form just to be on the safe side where the users had to tick that they were happy for their actions to be filmed and for the session to be audio recorded and other things, such as confirming they received the Amazon voucher as promised. If you would like to see a copy to amend and use for yourself you can download it here.
Let reception know that people will be visiting on the day.
This depends on the set up of your office and if you have a reception, especially in serviced offices. Provide them with a list of the people you expect to visit and what time they will be arriving. You should also give them your contact details, or your colleagues’ so they can contact you directly. You don’t want people running around the office asking if somebody is expecting Kimberly Wood whilst your in the middle of a study.
Have another team member help you out with logistics.
If you are going to be conducting the actual study like I was — make sure you’re not the one who will be contacted by participants to say they’re running late! I had to pause one of my sessions because a participant kept phoning to say she wasn’t going to arrive. Make sure someone from your team is available to pick up these phone calls and to greet people in reception, also to ensure people are turning up for the sessions. It ruins the flow of the study if you’re interrupted and you might miss some valuable findings — also it is rude!
Put up a do not disturb sign on the door of the room you’re using.
Let your team know that you will be running these sessions so if people are working close by to the room you are using, they aren’t too loud. Also put up a sign on the door saying that sessions are running. I had a colleague let themselves in to get a bottle of water from the fridge in the meeting room I was using during one of my sessions! Participants can lose their flow of thought and this can cost you some key insights.
I hope these tips will help you with actually conducting your first usability study. Just remember to be comfortable and enjoy yourself as you’ll be able to do a better job, and the participant will also feel more comfortable.
If you have any other questions or would like more information, then feel free to leave me a comment and I’ll be happy to help!