Redesigning AAC Through The Lens of Value-Sensitive Design

Collaborators: Julia Kispert, Anatoliy Cherednichenko, Ifraah Dhegadub, Andrew Nguyen

Figma Prototype

Our Goal: To make people more empathetic toward AAC users while making the AAC interface more customizable

AAC devices are commonly used by individuals with speech impairments. The users, however, often face issues such as a lack of empathy and independence. To tackle these problems, our Design Sprint team has designed a prototype that features a customizable avatar and incorporates AI to make communication flow smoother than ever. In a nutshell, this device aims to provide a personalized AAC experience for users while enhancing the understanding of their communication partners.

The prototype we develop offers a personalized experience for the user by allowing them to create their avatar and input their hobbies and interests to generate more customized vocabulary on their device. Additionally, users receive a tutorial on how to use the device, which is particularly useful for those who do not have access to pathologists and educators who can train them.

We also introduced a QR code on the device that directs people to a page displaying an introduction to the AAC user and their interests. The page also includes a simple AAC core board that invites the viewer to introduce themselves with an AAC board. This feature aims to promote empathy and awareness for AAC users and encourage more understanding in conversation.

The AAC device will be able to access the other user’s hobbies and algorithmically suggest conversation starters and vocabulary that the AAC device user might want to use.

We wanted to increase a sense of equality and understatement by those in conversation with an AAC device user. These changes were inspired by the Communication Bill of Rights:

Steps To Achieve Goal

The first thing we did was explore software that currently exists. We had no experience interacting with them, or instructions on how to use them. We struggled to express exactly what we wanted to say at the pace of an average conversation.

However, we realized that this may not be the experience all who use the device on a daily basis have. For this reason, as a group we also watched some videos on youtube watching people interact with AAC devices. There are many videos on youtube showing how users interact with different types of devices, and how long their response time is. On the other hand, these videos do not display the users feedback on the device.

The most vital part of the design process for us was listening to a panel whose lives are guided by the AAC devices. Whether they are a stakeholder because they use the device, or they are a stakeholder because they work or live with someone who uses the device, we gained a lot of value from hearing their own experiences. Specifically we were struck by the feedback from two users of the AAC device who described the lack of personalized vocabulary, identity, and recognition they feel when they use the device. Key notes from this panel include the following:

1. Lack of vocabulary is an issue. One female student spoke on how they love SpongeBob and playing uno, but there is not clear and specific vocabulary available to use in regards to both things. There is an option to program and personalize vocabulary, but the student described how it was difficult to do so, and she relied on her dad to help her do this. This in fact gave a sense of defeat and was time consuming.
2. All students involved in the panel described how people constantly talk over them. With the length of time it takes to create a response to a person’s question, people often do not wait or continuously get frustrated and talk over them.
3. One of the students described a loss of identity when using the device and a feeling of not being human.
4. They also talked about how they did not like to use the device. We wanted to find a way to make the device more enjoyable.

Now it came to creating a design. We had a whirlwind of ideas. When trying to narrow our ideas down, we wrote tons of problems and areas we want to change on sticky notes. Once complete, we organized these ideas into value oriented categories. We chose to take value oriented design as we wanted to prioritize the needs and values of the AAC community. Further, organizing our ideas into value categories allowed us to see a more organized approach to improving the device.

After looking at the whiteboard we chose to focus on empathy and respect. We chose to focus on these values as we felt they were most important in directly improving user experience with the device and in everyday life.

Once choosing values we began shooting out ideas.

1. Focus on the customization process. Doing this, we imagined that we could add an easy user interface that would allow users to download new vocabulary based on their specific interests. This interface would also allow users to input pictures for certain words. For example, for the word “me” it would be a picture of the user. Our goal here is to help connect the AAC device with its user identity, and also make students want to use the device more.
2. Build an AI that would personalize the AAC device based on a person’s interests and even reorganize the squares and present suggestions of what to say next in conversation. These presentations would appear as a shortcut, to also help cut down response time.
3. Building empathy. We talked about making an approach where someone communicating with a person using an AAC would also have to communicate through their device. By doing this we imagined a QR code on the back of the device saying “to communicate with me scan here” or NFC procedure where the other person in communication would scan the code and a message would pop up on their device describing that they are trying to communicate with someone who relies on an AAC device. This message would tell them a little bit about the person, what an AAC device is, and to be patient. The software will also give the option for the user to respond with a basic AAC device. If the user chooses this option, then they will better be able to understand the perspective of someone who uses an AAC device.

We understood that our ideas were cluttered and disorganized, but we wanted to tackle a little bit of everything.

Paper Prototype

We then created our paper prototypes. In these prototypes we accounted for everything in our third idea. However, we also included parts of our first two as we allowed a user to create their own profile.

However we did chose to not build an AI or attempt to describe the algorithm our prototype displays an example profile. In this prototype we only displayed the general functionality of a questionnaire we would like to be in our final prototype, that in theory an AI would use, to guide organization and vocabulary on an AAC device.

Figma Prototype

After our paper prototype we made a digital prototype on Figma. With this prototype we did cognitive walkthroughs and made sure all color combinations were accessible using the WCAG color contrast checker.

From the AAC device perspective we made a few changes:

• Made sure the boxes were all separated and had spacing between them on the AAC device to help those who struggle with motor skills.
• Made the buttons uniform, clear, and that a user can get to any page they need to at any time. The goal would be that a person could use the AAC device and not have to ask for support if they wanted to add photos or download new vocabulary.
• Made sure to give the opportunity to users to either take a selfie or create an avatar. If the project could continue, we would have added the avatar to many of the squares to depict the meaning of words, to help engrave a person’s identity into their device.

Initially we thought about having a chatbot interface for the phone perspective. We chose to do this to make it seem as familiar as possible in hopes to encourage use.

After much discussion and prototyping we decided to implement an interface similar to Spotify Wrapped instead. We found this design to be more engaging and more human than our initial chatbot interface idea.

While designing we reminded ourselves to have a similar style and format as the AAC perspective to make the perspectives synergize. The information inputted in the AAC perspective also transfers to the phone perspective which makes the process for both perspectives less tedious. This could be a way to give someone something to do while they are waiting for a response and may make the conversation more engaging.

We also chose to connect the device to Bitmoji for avatar creation. Though the avatar would not be created in house, with our time constraint on prototype creation we believed that the Bitmoji interface would suffice and could be brought into our software.

We achieved our final prototype!

We also drew a storyboard for our prototype, so that we could pitch it to other people and demonstrate the scenarios in which our design succeeds at.

Strengths and Weaknesses

Strengths:

• Hopefully with the addition of software to communicate directly to the AAC device, empathy between those in conversation will increase and there will be a decrease in someone talking over an AAC device user.
• An increase in personalization. With the addition of avatars or a selfie, and a questionnaire about common interests, the goal would be that once set up a user would not need help and would be able to have a more personalized experience.

Weaknesses

• User may need to have assistance with setting up the device. Further, with initial setup there will be a lack of independence which some may crave.
• Lack of testing. We were not able to directly test our prototype with someone who uses an AAC device. Us, the developers, do not have the same perspective and are not able to see the same problems as someone who would use the device everyday does. From here we would say it is ethically not correct to release the product until basic walk-throughs are carried out with someone who regularly uses AAC devices.

Conclusions

We learned throughout our exploration of different approaches to different values that it is difficult to address every issue. For example, some AAC users may value new features that require more setup such as personalized vocab or profile customization. However this would be at the expense of AAC users that value independence. If we tried to balance all the values our device would not excel in any value.

Perhaps the key is to make a design that is more versatile or to create multiple designs that specialize in different values?

In the future we would like to attempt to answer this question by creating one versatile prototype instead of our two prototypes that targeted different values.

Another lesson we took from this process was the importance of testing. Due to time restraints we are unable to have users test our prototype. Our ideas may be strong in theory, but not having the feedback from how others interact with our design, limits our learning opportunities. Next time, we will prioritize testing more to help us confirm whether or not our new features are favorable.