How “Smart” is Smart-home technology ?
- My goal in this smart home re-design, was to fix some pretty significant user flow issues, involving mobile settings options and a lack of breadcrumbs.
When engaging with the Amazon Echo, better known as Alexa, I quickly found myself getting stuck in tricky little corners of Alexa’s UI. From not having an easy way to sign in to your Amazon account, all the way to having zero buttons to quickly turn down TLC’s “No Scrubs” that you were just blasting (because your dad has claimed its the only way to test a speaker, and now its an unfortunate ritual you have brought upon yourself). I found myself asking, how “smart” is Alexa really, and what could be improved on? I decided to do a simple re-design and see if I couldn't resolve, or at least address some of the Echo’s sticky interface issues.
At first use of the Amazon Echo (2nd generation), I thought it seemed intuitive enough. It was able to hear voices in a fairly noisy classroom, and has a really sleek minimalist design. But upon further inspection and test interactions, my fearless group and I (somewhat fearful), found that it had some “settings” flaws and had issues creating a space for the user to occupy and move through. It was hard to track where you were in its menu because of the lack of visual breadcrumbs, and this was not made up for audibly.
Purpose
- We found that navigating through Alexa, it was fairly difficult to way-find, and the push from the device to get us to sign in on mobile seemed like a huge barrier to full access of the device.
My group and I noticed that the device lacks in some crucial areas of understanding those with accents or those who go back and forth speaking different languages, which fails to anticipate the needs of the user. We wanted to know if Alexa had any other language options, asked her to switch to Spanish and were promptly given a reply back stating that we had to log into a connected device to manually switch her language settings, this was also a mobile device that we did not have access to, which could be a common problem in most households. We thought that for many people who speak more than one language this is a frustrating challenge with the device. Ideally, Alexa should be able to understand a user switching between languages. She could then respond in the language she was spoken to in. We also researched how she responds to accents and how well she can understand them. We found that she actually does a pretty poor job at recognizing accents. With this type of technology, Amazon can’t just aim their product at a generic and stereotypical American “western accent”. To be fully inclusive and user-friendly, Alexa must be able to be more intuitive with understanding different accents, inflections, and be prepared to switch actively between languages, as to follow the user’s needs. This login block was a big problem we found for almost any personalized task on the Alexa device. Changing music stations (if it was a paid service: Spotify, Apple Music, etc.)required the user to login to their mobile amazon device to allow the process. We also found that along with this navigating through all of these “pages” we had a hard time understanding where we actually were in the interface, there was little sense of direction, and we found it fairly difficult to way-find to where we wanted to be.
Process
- My hypothesis was that participants will struggle with account issues and be confused with navigation using standard Alexa. The test device will offer option to speak or enter in on screen a 4-digit pass-code that will bypass account login.
In our Amazon Alexa study group, we threw around some ideas for what we thought could be useful improvements to further support the user. Another one of my teammates mentioned having a simple code to put in that would save to the device, rather than to reroute the user to a totally separate device that they may not even have access to. I thought that having a screen would also help support this ability to login, and would open the device up to those who wanted or needed something more visual. I also wanted to be able to add a quick volume option since the only physical volume changing feature were buttons that were just somewhat slow. My official hypothesis was that the participants will struggle with account issues and will be confused with navigation issues using the standard Alexa; using the test Alexa with a passcode to bypass account login from mobile app and screen interface will help the participant better navigate the device.
So for my testing I knew that the main things I wanted to “upgrade” and then test on the Alexa device were the login settings issues, which would then assist with tasks like changing language settings or playing music, integrating a small touch screen, to improve wayfinding and navigation, and finally adding a turning dial volume feature. I created a few simple tasks that would ask users to interact with these features and then I would watch and document whether or not the user struggled with what I had hypothesized they would, and then how they would navigate around the problem.
In my Usability Test Plan, the main objective of the study was to test approximately 2–3 participants in demonstrating use of 3 specific tasks with the Alexa device. The tasks seem simple and should be easy for the user to accomplish, however the user should come across a few issues in navigating through the questions with the standard Alexa (without any modifications) vs. the test Alexa (with modifications to interface).The participants will run into issues with having to immediately be navigated to logging into an Amazon account and making adjustments to Alexa through the Amazon Alexa mobile app with the standard Alexa device. By using a login 4 digit code the test Alexa will however be able to navigate around these issues and keep the user only interacting with the one device. The participant will also come across the issue of not having a volume dial on the device, as well as not being given auditory “breadcrumbs” to assist in navigation, which the test Alexa will have on the visual interface on the top of the device. The interface that was used to represent the test Alexa was simply a vector-created image. It would show basic home screen options like time, date, notifications, etc. Then on the music menu it would show the song/artist, time, a home button and a back button (to keep the user from having to go home for each task), as well as having a turning volume dial.
Prototype of New Alexa Screen
I then created a script from which I would follow to run my Usability test with other students in class as participants. I did tell participants up-front that they would have to suspend disbelief for the purposes of this test, the test Alexa would be only represented by paper, and I would personally be reading the device’s script.
The Script:
- Asking participants to interact with navigation, and find a way to login to Alexa. Test was done twice, once with the standard Alexa, then with upgraded prototype Alexa.
(Will be read twice. Once with standard Alexa, then once again with test Alexa)
Facilitator: “Can you ask the Alexa device to adjust it’s language settings?”, can you ask it to switch to Spanish?”
(Participant asks Alexa to switch language settings.)
Facilitator: “Can you ask Alexa to play music?”
(Participant makes command for Alexa to play music (nothing specific).
Facilitator: “Can you manually adjust the volume to turn the music down?, do not audibly tell her to turn the music down.”
(Participant will look for controls to manually adjust the volume. If they cannot complete this, allow the participant to tell Alexa “Stop.”)
Facilitator: “Can you ask Alexa to go back to her home page?”
(Participant will ask Alexa to “go home” (or some variation). Monitor where Alexa device believes “home” to be from this point in her interface.)
(Tasks are completed.)
I found from testing this script twice with participants (once with the old Alexa and once with the test Alexa, that participants generally did not know what to do once a login block was reached and then just tried to navigate back home. Having a 4 digit numerical pass-code option though that could be said or typed in on the screen was said by participants to be very helpful.
Outcome
- Participants were surprised that there wasn’t already a way to bypass this login option, they agreed that a screen would be the most efficient option in assisting navigation. Participants agreed that they would rather put in a 4-digit code, than have to sign on an external mobile app. Since the point of a voice device like this is to be “hands-free”, requiring the user to have their phone in their hand seems counter-intuitive.
Looking at the data from the Usability Test, it was very obvious that when communicating with the old Alexa participants were confused by the redirection to the mobile login. Most were surprised that there wasn’t already a way to bypass this login option to keep everything directly on the Alexa device. I chatted with each of my 3 participants after the tests were conducted very casually about what they thought and what surprised them, none of them having any previous interaction with any of the Amazon personal voice devices. They mentioned that they were also somewhat shocked that changing something as simple as a voice setting was not a quick 2 step process with Alexa. Obviously Amazon is taking privacy as huge priority with this blockage and re-route to the mobile app, but the participants agreed that they would rather simply put in a code, that would help them get access to “locked” features almost instantly. Since the point of a voice device like this is to be “hands-free” requiring the user to have their phone in their hand seems counter-intuitive. They also mentioned that having a small touch screen would be useful, they wondered if the price of the device would be affected in the real world, but thought that for users that needed a more visual representation of a menu that a screen would be the best and most efficient option. They agreed that a screen would definitely help with navigation and showing specifically where the user was and gave them options to quickly get back to a previous page or home.
Reflection/Introspection
- Overall the testing was effective and the feedback confirmed that Alexa does have some usability flaws with navigation and login options. Re-testing would have allowed me to adjust the way I administered the voice features of Alexa to add realism.
Overall I think the testing was effective and the feedback I received from the 3 participants was very useful and insightful into what was and wasn’t working for the Alexa Echo (2nd gen) device. I would say that if I was able to go back and revisit some of my work I would try to for one have another individual say the Alexa lines in the script, or I would have this pre-recorded, either way the participant must suspend disbelief slightly to work within a test like this, and for the most part the participants understood and adapted and responded fairly well, but I do have to consider that these were UVU design and development majors that already understood the Usability Testing scenario and were most likely more tech savvy than most other non computer science majors. With this being said, having done this same testing with any other demographic this scripting format could have been more confusing. The main project challenge I faced was finding ways to update something that already had so many iterations. The amazon Alexa devices are released quickly now and the options and features are vast. I would get stuck at times looking at all the specifications for the Amazon dots, spot, and echo, with all different generations. And of course all of them had different strengths and weaknesses and I wasn’t able to interact with all of these devices physically to test them, so I reminded myself to focus on the one device at hand and its specific generations specs; knowing the device well and giving myself constraints actually allowed me much more creative freedom in brainstorming possible solutions specifically for the Amazon Echo 2nd gen device.
Christian Mercado is a student in the Digital Media program at Utah Valley University, Orem Utah, studying Interaction Design. This article relates to the “Smart-home Technology” project in the DGM 1240 course and is representative of the skills learned.
