Usability testing for beginners

Things beginners should learn (nay) know about Usability Testing in the startup environment.

Pramit Singhi

Published in

Bootcamp

10 min readOct 8, 2020

Part-1 of the series ‘Product For Beginners’

In this series, there will always be two sections:

My learning — About my personal notes in crisp manner
My test —About my journey of the test

Now I know, the ideal way is that I should have written about my test first and then my learning at the end. But the reason, for me to start this series was to maintain my notes/insights. If ever, you want to revisit the learning, you shouldn’t have to scroll/skip my test section. See different perspective, you are gonna see a lot of it in the coming series. That’s what makes us humane.

So, if you find something not appropriate or right, feel free to comment or correct me. The goal is to just have the best concise learnings around.

My Learning.

Usability testing is a kind of experiment performed on the target users for a crafted product to gain insights about the usability of the product from user’s perspective. Thus feedback from this helps to improve product experience. Ideally, should be done before a product is made available for all.

🤔 Why it is an experiment?

It is an experiment because it is used to test a hypothesis (hypothesis being the created product experience). As there is no one fix method, you get a chance to be very creative around the procedure to perform.

There are many methods to perform a usability testing. It can range from simple to complex. It depends on the kind of product, the insights required (targeted metrics) and targeted users.

For example:

If it’s just a small feature in a digital product, you can create a prototype and test it out with targeted users observing their behavior and their questions.
If you have changed copy in a redesign, go out and check it out with your users. Trust me on this, people are eagerly waiting to give you different perceptions of a single line.

PS: In a little exp in office, we figured 3 interpretations of the same line regarding a currency detail. Finally closed on one. But this exp surely helped me to understand which words to not use specifically.

✍🏻 What’s the ideal structure of process?

Figure out the users and targeted metrics [Product Dependent]
Finalize a method and the tasks in terms of the user flow as per experiment [Product Dependent]
Print and prepare the docs and mock-ups/ prototypes [Templates]
Call and inform around 8–10 targeted users — very important; be professional and book times in intervals [Keep Buffer]
Perform a mock-up test to check everything
Prepare the user — very important; [Give Context]
Perform the test
Analyze the observation — [Gather Insights]
Present the problems to stakeholders

After presenting it to anyone, you can have heart-full discussions with PM, Designers and others about what should be the right solutions to implement.

📄 Good to have docs

There are certain documents which are always helpful for conducting a smooth Usability Testing

Introduction Script — A collated sheet of all points to be informed to the user before starting experiment. It helps to convey same information to all users and not slip up anything else which affects the experiment results.
Task Sheet — It comprises of all the actions to be performed by the user. It is given to the user to help with the actions expected to be performed. It is not a broken down step by step thing. It is a higher level task sheet. Not required if there is just one task.
Observation Sheet — It is filled when the user is performing the actions. It contains the observation about user at different steps in actions. It is maintained by the observer.

🕹 How to prepare the user?

The experiment with the user should always start on the time. Break the time given to a user into 3 segments:

Introduction Time — Explain the actions, give them Task Sheet, ask the user to speak their mind and do not feel any complex
Experiment Time — Help if the user is stuck somewhere but do not lead the user through any action or words
Post-Completion Time — Get the user to write/discuss their thoughts about the action; template can vary but just gather all their thoughts

During the buffer time basically between different users take a break; prepare the setup for new user; arrange the docs and remember your script. Keep the body language more receptive and not appear dominating at all. Make them comfortable around you.

📊 What are the insights?

When a product is created/ some feature is introduced/ or any change is made; there is a fundamental impact assumed because of it. We call it metrics (generally it’s quantified to tell you accurate measurements about how the process is functioning).

The insights are related to the metric which is to be improved. It’s connected because the tasks for the user are decided on the basis of metrics.

Therefore, insights are the results from the observations sheet obtained from the different tasks performances. These help to identify the user mind set with respect to the product experience.

Whatever be the method of exp, do not forget that the aim is to gain insights around the experience of the crafted product from the target user.

⏳ When should this be performed?

Ideally, this should always be performed before a product development starts. So that when a certain solution is crafted for the target users and the product is seem to expect the way it is working then you are get more affirmation about the solution. It helps:

To gather data and affirms that the solution works in the expected manner
Find out small bugs, like being around the copy (text) etc.
To reduce the tech team effort as they don’t have to reiterate

But, if for some reason, not able to perform it before a launch due to the fact of “ launch-learn-iterate-repeat” proverb, well then include it anytime before the next phase release.

Please do not get this confused with UAT. The test performed after development is UAT (User Acceptance Testing) one of the last stages of the software development life cycle. It is performed after the software has been thoroughly tested. It has a totally different motive.

My Test.

When I started out, I had no knowledge as how to perform a usability testing. All I knew was what a Usability Testing is and heard about just one method where you set up camera in a room and observe a user performing the task. I found it very complex. I always felt people get conscious around the camera.

My project was on the Zoom eScooters (India’s first electric dock-less shared micro-mobility solution). The biggest challenge I felt was that to perform UT for a product which just didn’t has just a digital part but also an on-ground service. This was also my first project around Research, so I was equally motivated and afraid. But anyhow, I started.

⏱ Research for the Research

There is an infinite pool of content on UT available online from various sources (designers, design studios, research organizations). I was going through lots of content (articles, books, case studies and research papers) in a day around:

What exactly is UT in it’s core? What are the different kinds?
What is the role of the user? What is the ideal number of users? Does it depend on the kind of experiment? How should I select the users? What criteria should I keep to select them?
How do I align a UT to my project?
What should be the scope of the project?
What metrics should I go for? What are the Product values?
How should I break different tasks? What should be the right questions? What method should I follow?
How should I structure the UT as per project timeline?
How do I quantify observations? Should it be qualitative or quantitative?

Some personal notes about usability testing, structuring, parameters, how it affects eScooter etc.

In a learning phase like college all the reading seems right but when you are doing a project in a company, you can’t waste all of the time reading. Your task becomes figuring out the problems fast and start acting on solving them. I was worried because I was not getting any clear answer, lot’s of perspective to look into, afraid of not meeting timeline, project pressure.

Honestly, I couldn’t find any article which could satisfy all the questions and answer me how to proceed as a beginner for UT. My manager/mentor after seeing me all drowned into content and confusion passed me this tip: It’s actually better to fail fast than learn slowly.

🙌🏻 Meeting with a User Researcher

While I was struggling for a proper direction to plan the UT, my manager arranged a meeting for me with an experienced User Researcher, Sajal Nagwanshi from Flipkart. Our conversation led to a few great points around planning a UT:

Always define the scope of the research as a clear cut statement, like project mission.
To identify the goal/objective of usability testing — We can’t cover a lot of different things in a testing.
Start small and fast. To not wait too long to conduct experiments for a large audience. It was more about quick experiments, iterate and repeat.
Decide on quantitative vs qualitative testing based on the scope of the project.
Select the users based on the use cases and objectives of the product.
Don’t make it complicated. It’s just people and you.
Present the results to the team from the perspective of the users, show their confusion and emotions as a result.

👟 Ready, Get-Set, Go

Now, I had a clear idea around structuring usability testing.

Figured a clear scope of the project, which things and why.
From the heavily quantitative testing idea, I shifted to qualitative testing idea with a small set of users.
Users were selected from the different personas and the constraints they face were kept in mind while developing the prototype and tasks.
Discussed on product values and flow which is to be prepared for a user. The product values help to write the tasks for the testing.
Created the plan of UT with different secondary actions (around recruiting users, mock-up experiment, testing, analysis etc) It is also about what time the users will come in, which scenario to give, how the user will go about it and how to record the details of testing.
Figured there are no right questions but there are lots of ways to avoid asking wrong questions like leading questions, putting an idea in the question. There is also a suggested way to stay as quite as possible in the testing and let the user speak more.
Keep the body language very receiving and not dominating, which should let other people be more open about their opinions to you and just listen to them performing actions.

🕶 D-Day

The day when the users were called in and UT was performed. As the product caters to B2B and B2C, so users were from both the groups. Their tasks were almost similar but the second group had more features so the tasks were a little different.

Performed a mock experiment before the first user to open up and as a warm-up practice.
Video camera and screen recorder were set to record all the emotions and conversations with the users
Test eScooters to go out on a ride with the users
Called in users for the testing, gave them context and introduction
Handed the tasks sheets and filled in observation sheets
Performed a full digital and on-ground UT.

It was complete fun. Everyone should definitely try this once.

After this, I calculated insights related to the tasks and metrics in Excel and had a very insightful presentation with the stakeholders. Overall the product app experience was perfect whereas we improved mostly on the ground operations. We were able to provide an amazing product experience and Usability Testing helped to affirm us with the solution crafted.

I can’t share the data around the UT, but feel free to ping me to chat more on the doc templates, introduction script, different methods to try out etc.

I had a lot of fun learning while doing this activity and now I have been handling Usability Testing at Zoomcar. Personally, I have completed Usability Testing for more than 3 projects and every time I find something to improve either better method, more prepared docs.

I came out of my fear, confusion and doubts. I genuinely feel that talking about the problems you face helps you learn a lot, especially in the product/design field.

Anyway, I hope you enjoyed reading this article and always remember this amazing quote :