Phase 5: Evaluating the Prototype
This document is written for the Midterm Project in H541: Interaction Design Practice — Instructed by Sonali Shah, at IUPUI in Indianapolis, IN.
Other parts of this project:
The goal of this document is to evaluate the design presented in Phase 4: Dynamic Interactive Prototype. In a “real-world” situation with this project, the iterative design process would permit the revision of the prototype following this feedback. For the purposes of this class, it will be the final evaluation, a post-mortem of sorts, to put a cap on the project.
In 5.1: Usability Test I outline the parameters and goals for a “Quick-and-Dirty Usability Test,” as described by Leah Buley in The User Experience Team of One (Buley, 2013). This is followed by the results of the usability test in 5.2: Findings and Analysis. Finally, I extract the essential direction taken from the testing that I would theoretically apply to the next phase of design in 5.3: Design Recommendations.
5.1: Usability Test
To conduct the usability test, I followed the guidelines of the “Quick-and-Dirty Usability Test” as per Method 21 in The User Experience Team of One (Buley, 2013). With this excercise, we “forego rigor and perfectionism” and “test the designs with anyone who’s available…[and] unfamiliar with the product.”
In a less formal and detail-oriented context, this exercise could be used often and regularly to see how well the current design aligns with goals.
For this project, I investigated the prototype with 2 subjects. I started by giving a concise description of the app.
“It allows you to collaborate with teams to complete projects and store all your personal and team files in the cloud.”
To begin, I asked them to attempt three key actions. I asked them to think aloud, so as to reveal their thought patterns and provide more insight into difficulties (Preece, 2016).
- Try to navigate to where all your files are stored and then upload an image from your phone’s camera roll.
- Try to share the image you just uploaded in the conversation with all members of the “ABC Project Team.”
- Try to view only your unread notifications.
Each of these actions represent a core functionality of the app available in the prototype: uploading files, sharing existing files with a team by name, and managing notifications.
I asked my participants if I could record our conversation, in order to reference it later for transcription as I was unable to find anyone to take notes on my behalf. This was good practice for working with what is on-hand and gathering participant consent.
5.2: Findings and Analysis
My first participant was a full-time student, age 23. Below is an abridged transcript of the conversation.
Uploading a File
- “Uh, I’ll go to files obviously”
- “Obviously add”
- “Obviously photo library”
- “Okay fine this photo is fine”
- “Oh now it’s uploading”
- “That was easy”
Sharing an Existing File with a Group Chat
- “I go to Teams”
- “ABC Project Team”
[Presses the add button on the Team Overview page]
- “Oh, wait. I messed up.”
- “Okay, so team chat, then I’m gonna add it here”
- “Press the plus button”
- “Pick the one that I want to share”
View Unread Notifications
- “I would go to notifications” (bottom tab bar)
- “The blue ones are unread”
[Hits the sort by button rather than the filter by unread button]
- “But I want to sort by unread”
- “Oh I see, maybe this checkmark button”
[Hits the mark all as read button]
- “Oh no! What happened?”
[“All your notifications are marked as read now”]
- “Oh no! That’s really confusing!”
[I reset the state of the app]
- “Oh yeah, just this other button. Well that’s confusing”
[“This button is used in the iOS Mail App to sort by unread”]
- “Well I wouldn’t have known that was there either. You need to have something that does a better job of telling those two apart.”
I gathered two important pieces of information here:
- Both sticking points that Participant 1 encountered were related to problems with clarity. Each time, it boiled down to: “what can I expect to happen when I interact with this button?”
- Some actions are somewhat destructive (such as altering the state of multiple notifications from “unread” to “read”). The feeling that actions are dangerous can contribute to a more hostile and frustrating experience.
My second participant was an attorney, age 50. Below is an abridged transcript of the conversation.
Uploading a File
- “I tried looking into one of the files here um, it says files so I’ll go to files”
- “Then I’ll go to … so I’ll go to task management”
[Opens a document in files]
- “Oh, nope that’s not it”
- “Okay back to all my files”
- “I’m opening a client file”
[Tries to click on another document]
[I provide him some more guidance on the goal of the action]
- “Okay then I’ll go to plus”
- “Then I’ll choose ‘Photo Library’”
- “Then I’ll pick a photo”
- “Woohoo I did it!”
- “I don’t know that plus is where you’re gonna add a file”
- “It would be helpful if it said “upload file” or somehow gave some instructions”
- “But it would be easy to remember if I’ve done it once before”
Sharing an Existing File with a Group Chat
- “Okay I’m going back to teams”
- “I’m opening ABC Project Team”
- “And add new file”
[Tries to hit the plus button on team overview]
- “Well, I guess that’s not it”
- “I’m gonna hit the information button to see if it gives me more info about what I can do with the file”
- “I can’t find what this should be”
[Gets stuck for a bit; I provide a bit more guidance by reiterating the task]
- “I suppose it’s intuitive to go to the team and then get into the chat”
- “I guess going to the plus makes sense”
[He finds the plus button and I explain that in the prototype, searching for the file name brings up options]
- “Seems like there should be a way while you’re in the chat to access all the folders and find what you’re looking for”
[“Kind of like a file picker on the computer?”]
- I’m not sure what that is
[“Where you can browse for any file that’s saved on your computer”]
- Yeah, just like that
View Unread Notifications
- “Alright well I’m gonna click on the button that says notifications”
- “I would hit this”
[Hits the “view unread” toggle button]
- “That’s great because it’s just like in the Apple Mail app. So that’s really intuitive for me.”
- “It feels like you need to add something. Maybe you need some more descriptive words on some of these.”
I gathered several insights from my test with Participant 2:
- Using the same symbol for different types of actions introduces confusion. This was also the case with Participant 1. For example, the plus icon is used on the Files screen to mean “compose a new document” or “add an existing file from elsewhere.” It’s also used on the Teams Overview screen to mean “create new team.” And it’s used on Team View screen to mean “create new conversation, file or folder.”
- In contrast to Participant 1, the “view unread icon” is highly recognizable and its behavior is predictable for those already familiar with the icon (especially Apple Mail app users).
- Actions and buttons often make sense — but only after they are discovered. Without introduction, there is a higher likelyhood that they will not be used, which can contribute to lower functionality and a more frustrating user experience.
5.3: Design Recommendations
In order to address the takeaways from the usability test, I would make the following changes in my theoretical design iteration:
- Increase clarity. This is primarily in terms of button purpose. By adding text labels in some places, the clarity of what action is tied to the button could be significantly improved. There’s a balance to strike between simplicity and easy of use, but the current design certainly airs on the side of simplicity and more specific buttons could greatly improve clarity (Byttebier, 2015).
- More specificity. As mentioned in the takeaways for Participant 2, the same symbol is used for multiple actions, in more place than one. All of these are distinctly different actions. Their associated buttons should provide much higher transparency into the action that will happen when they are pressed. By creating more specialized and granular buttons in places like the “+,” user understanding will greatly increase.
- Create onboarding. Introduce new users to potentially complicated actions by educating them with informational overlays. Once discovered, these actions will become much easier to repeat.
- Improve safety. Certain “destructive” action, such as marking all notifications as read, need a confirmation dialog. This would inform users of the consequences of the action (all notifications will be marked as read and this cannot be undone), and required to confirm. This introduces a higher level of safety by requiring the user to stop, think, and understand their actions. Improving safety should also increase discoverability throughout the app. If users feel that their actions are safe, they are more likely to explore.
In this document, I have outlined the results and analysis of usability testing on the high-fidelity prototype. Even with such a concise study, I was able to draw a significant amount of insight. Many of the challenges boiled down to clarity. By providing more explicit labelling and instruction, the app could become much easier to understand and use.
My only lament for this part of the project is that we do not have to chance to look at more in-depth aspects of usability testing. This would include not only learnability, but also effectiveness and memorability (Preece, 2016). These factors are difficult to test with a visual-only prototype. They would be much easier to study with testing on real users of a built and deployed product.
One of my primary influences for this project was the collaboration app Quip, which I love for many reasons. But I also feel frustrated that it struggles with organizing documents and chats by team — it seems to assume that you only work with one team of people. It also has a bizarre way of organize files between private and public, while also allowing you to add external files. This still confuses after more than a year of using it every day.
My hope for this project was to paint a picture of an app which captured the marketshare of team, document, and file collaboration. While the prompt guided me towards the mobile interface, I would love to create a desktop interface in the future as well.
The process has taught me a great deal about understanding users, conducting research, and interating based on real-world requirements — all through experience!
- Buley, L. (2013). The User Experience Team of One: A Research and Design Survival Guide. Brooklyn, New York: Rosenfeld Media, LLC.
- Byttebier, T. (2015, March 23). The best icon is a text label. Retrieved October 14, 2017, from https://thomasbyttebier.be/blog/the-best-icon-is-a-text-label
- Preece, Rogers, & Sharp. (2016). Interaction Design: Beyond Human-Computer Interaction (4th ed.). Hoboken, NJ: John Wiley & Sons, Inc.