Soundcloud — Usability Testing (Part 8)

A fundamental step in the UX design process is validating your designs. Usability testing is imperative for this validation. It involves measuring characteristics of a users’s interaction with a product. The main aim is see how well users can complete specific tasks and to identify frustrations and problems encountered while doing so. In this post I will discuss usability testing, how I conducted 3 tests and the results from the testing.

What, why, where, when

ISO 9241 described usability as the “effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments”. Effectiveness and efficiency can be are measured quantitatively by recording the users ability to perform a series of tasks, and satisfaction is measured after each task with more qualitative, attitudinal data, by asking the user to rate their experience on completing each task (Preece et al, 2015; Nielsen, 2012).

It can be carried out throughout any UX project to obtain different insight depending on where it is used. At the start of a project it can be used to explore new opportunities by evaluating how users interact with the current system or a competitor’s system, in the middle of a project to validate and test prototypes, and at the end of project to determine if the product works the way it is intended.

They can be moderated or unmoderated, conducted in a lab, in context or remotely. The choice of method to use will depend on access to users, resources, budgets and timeframes. Remote testing can be quicker to setup and easier to find participants but moderated testing performed in a lab or contextually provides more control and better insight. This is due to the facilitator being able to read the user’s body language and know when to probe or ask follow up questions, guide the participant and keep the testing on track (Nielsen, 2012).

What to test:

The tasks I’ve selected to test are to validate the new functionality related to filtering, sorting and editing(information architecture and content strategy) identified in my previous posts. By observing users perform these tasks I will also be able to assess the usability of this functionality and flow of the redesign. Below is a list of the 5 tasks:

Task 1: Change your name from Jesse J to Jessica J
Assumption: Users will be able to navigate to the their profile and change their details with ease.
Task 2: Go into a playlist called Dark Eyes by Half Moon Run and remove the track called Full Circle.
Assumption: Users will be able to navigate and understand the layout and functionality of the redesign
Task 3: Go into your saved tracks and sort them by A — Z.
Assumption: The sort by feature easy easy to find and access
Task 4: You want to make a playlist available to listen to offline (without internet.) Make the playlist Dark Eyes available to listen to offline.
Assumption: Users will be able to use this the new feature without effort
Task 5: You want to find tracks that meet a specific set of criteria. Using the application filter tracks so you see the following
- Dance & EDM music, House music
- Posted this week
- From France and Germany
Assumption: The Filter functionality is easy to use and appealing

Conducting the Sessions

For the purpose of this project I performed 3 usability tests. In a real project it is said that you can get the best results from testing a product on 5 participants. Participants should represent your chosen personas. I created a script, release form, pre-test questionnaire and task sheet to aid with the testing. I used the script to brief each participant on what to expect from the testing session, what they need to do and to answer any concerns they might have. The release form was used to get permission to record the session. I reassured each participant that the recordings will only be used for research purposes and that no one else outside the project would see them.

The pre-test questionnaire was used to gather some related information about each participant to envisage how that user thinks while using the application. In a real UX Design project this pre-test questionnaire could be more detailed and specific to the key areas identified in the research phase.

I used the task sheet to guide each session and keep each test on track. After each task I asked a series of questions to gain a better understanding of how the user performed.

See this link for access to script, release form, pre-test questionnaire and task sheet.

I used an application called AZ Screen Recorder to conduct each test. This is a great application to use for testing and I thoroughly recommend it. It offers functionality that allow you to record the front camera on your phone, visually displays touch gestures and has no limit to the length of video you can record.

AZ Recording of User Testing (Scrrenshot)

The Results

Participant 1

Female (37), right handed, holds pone in left hand and uses right to touch screen. Owns an Iphone. Uses Itunes mostly. Favourite applications include Whatsapp, Pinterest, Instagram and Google Maps. Dislikes Facebook and Linkedin. Has reading glasses but never wears them.

Summary

Task 1: Change your name from Jesse J to Jessica J

Participant easily navigated to user profile page and found edit functionality. Had no issue with changing the name. She initially expected to see a save button but quickly assumed it saved automatically. She found the task easy to complete and quite standard compared to other applications she had used. No critical blockers.

Task 2: Go into a playlist called Dark Eyes by Half Moon Run and remove the track called Full Circle.

Participant navigated to the correct playlist with no difficulty. Task was completed with ease. Note: Initially when entering list of playlists participant expected to see a local search option on page. “if this was a long list of songs or playlists I’d expect a search bar so I could filter what I was looking for”. Although she raised this as an issue verbally she had no difficulty finding the playlist and completing the task (what people say vs what people do).

Task 3: Go into your saved tracks and sort them by A — Z.

Participant hesitated after clicking My Music. I think this was due to the wording in the question asked. The question says “Go into your saved tracks” but on the prototype the it says “Liked Tracks”. This is an administration issue and not considered to be an a usability issue. In re-writing the tasks in future I need to make sure terminology is consistent with that on the prototype. Participant completed task with no difficulty.

Task 4: You want to make a playlist available to listen to offline (without internet.) Make the playlist Dark Eyes available to listen to offline.

Participant completed task with no difficulty. Liked that you could save tracks to listen to offline.

Task 5: You want to find tracks that meet a specific set of criteria. Using the application filter tracks so you see the following — Dance & EDM music, House music — Posted this week — From France and Germany

Participant went to search page to find filter. Upon entering page hesitated briefly before seeing and clicking on Floating Action Button to Filter. Commented on no liking the FAB button to filter “i don’t like the this button down here”. Had no difficulty and made no errors in completing task. Comment: “oh i like this. It makes it easier to find what I like”

Participant 1 Summary

Participant liked how easy it was to get around the pages. Liked the left navigation and filter panel functionality. She didn’t like the branding as it reminded her of when she worked for the phone company Meteor.

Participant 2

Male (33) right handed, sometimes uses two hands. Phone — Xaomi smartphone. Uses Spotify and Youtube. Likes Spotify as it has so much music. Dislikes playlists on Spotify as they don’t give what you want …“Playlists are s**t, when you don’t want to look for something you stick on a playlist but the songs are always bad”. No disability.

Task 1: Change your name from Jesse J to Jessica J

Participant got to edit screen and changed users name. Expected to see a save button and looked around the page for one. Assumed eventually that it saved automatically but was unconfident to click the back arrow. “I like having a save button as it gives me the comfort of knowing it saved.”

Task 2: Go into a playlist called Dark Eyes by Half Moon Run and remove the track called Full Circle.

Participant tried to click and hold track to get option to remove track. His expectation was that the functionality in the 3 dot menu would appear on screen. After this didn’t work he was able to complete task with no difficulty.

Task 3: Go into your saved tracks and sort them by A — Z.

Participant had no difficulty completing task.

Task 4: You want to make a playlist available to listen to offline (without internet.) Make the playlist Dark Eyes available to listen to offline.

Participant completed tasks with no errors or difficulty.

Task 5: You want to find tracks that meet a specific set of criteria. Using the application filter tracks so you see the following

  • Dance & EDM music, House music
  • Posted this week
  • From France and Germany

Participant had no difficulty completing task. Liked this feature as thought it could “fizzle out all the stuff you don’t want to listen to”.

Participant 2 Summary

Liked the layout and colour scheme. Liked the explore page and understood all elements on the page. Had no major difficulty completing any of the tasks and seems to learn how to use the interface immediately. Didn’t really comment too much on the differences between the Explore page or Stream page so maybe it is not clear what the user is seeing.

Participant 3

Female (27), right handed, Samsung Prime. Uses Spotify. Likes Spotify as it’s simple to use, links you to new music, categories everything and makes cool playlists. Favourite applications include Facebook, Instagram and Whats app. Dislikes Snapchat as doesn’t know how to use and thinks it makes no effort to help you. Participant did not want to filmed.

Task 1: Change your name from Jesse J to Jessica J

User tried to click on the display name but quickly seen the FAB button to edit. Was able to change name but expected to see a save button at bottom of page or using the button on the keypad to save it. “i kind of thought if i click the back button it wouldn’t save” Slightly unclear at the end. Liked the layout and colour scheme

Task 2: Go into a playlist called Dark Eyes by Half Moon Run and remove the track called Full Circle.

Participant had no difficulty finding track and removing it from playlist. Found it easy to complete and straight forward.

Task 3: Go into your saved tracks and sort them by A — Z.

Participant had no difficulty completing task. Liked this feature “yeah that’s really cool, i like that. because when your going through your tracks on other applications its not always catagorised that way.” Expressed interest in using this feature. Found task easy to complete.

Task 4: You want to make a playlist available to listen to offline (without internet.) Make the playlist Dark Eyes available to listen to offline.

Participant had no difficulty navigating to page or completing task. Expressed interest in feature “That’s good so the whole album is now available offline”. Said she would use this feature. Found task easy to complete.

Task 5: You want to find tracks that meet a specific set of criteria. Using the application filter tracks so you see the following — Dance & EDM music, House music, — Posted this week, — From France and Germany

Went into wrong category but easily corrected herself. Midway through task expressed interest in feature “this is really cool”. Understood after applying filter what the number 5 meant. Found task easy to complete. Found feature to be very different from any other music applications she has used. Really liked the idea of using the filter panel.

Participant 3 Summary

Quickly became very familiar with layout and structure of pages and navigation. Very clear and likes the colour. Understood what recommended and trending meant. Thought that top 20 was top 20 tracks you’ve listened to and not a chart of the top 20. Trending and Top 20 are perceived to be the same “cuz if you’ve got trending i don’t see why you’d have top 20 as well”. Likes the side navigation. Uses the music she’s stored more frequently. Likes how My Music is laid out. Very easy to read. “I actually think it’s clearer than other application as there is only 3 options Like tracks, playlists and artists.”

Key finds from User Testing

Overall the navigation was easy to understand and interpret. All participants had no major difficulty completing any of the tasks. Interest was expressed in using the Filter panel and sort by to look for a order tracks.

There was some minor issues with saving edit profile details. In the next iteration I plan to change the back arrow to the word SAVE on the edit profile details page. This will eliminate any confusion users have over whether or not edits have been saved.

There seemed to be a lack of interest or confusion over the content on the Explore page and it’s elements. The layout and content on screen needs to be revisited to engage the user more so.

Although users were able to access the filter panel it exists in 2 different places on screen which might be confusing. One position should be chosen and persist across all screens where it is available.

Mistakes, mistakes, mistakes

I made some mistakes conducting the user testing that if i was to do it again would be mindful of.

  1. Ensure the recording software is setup correctly: For two of the recordings I forgot to turn on the mic. As a result I did not record what participant 1 and 2 said. Luckily I wrote detailed notes while conducting the sessions and analysed the data straight after the recordings while it was fresh in my head.
  2. Use ratings: Although the post task questions provided good insight to how the participants performed, in future I would use a rating scale for these questions so data could be analysed objectively. One persons opinion of easy and difficult may be completely different from the next. A scale can more accurately assume consistency.
  3. Don’t Lead: I found I asked some leading questions on occasion. With the third participant I asked a leading question for task 5 which made it easier for participant to complete task. To get more realistic results I’ll more considerate with the structure of questions asked and answers I given to participants during testing.

What’s next?

In the next post I will create a video revising the full process carried out in the project including the information from all precious posts. I will also reiterate the design based on the findings from the user testing above and make recommendations for further research.

References

ISO 9241­11:1998 Ergonomic requirements for office work with visual displayterminals (VDTs) ­­ Part 11: Guidance on usability. Retrieved Feb 15, 2016 from https://www.iso.org/obp/ui/#iso:std:iso:9241:­11:ed­1:v1:en

Nielsen, J (2012) Usability 101: Introduction to Usability retrieved January 5, 2016from https://www.nngroup.com/articles/usability­101­introduction­to­usability/

Preece, J., Rogers, Y., & Sharp, H. (2015). Interaction Design: Beyond human-computer interaction (4th ed.). John Wiley & Sons.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.