Social App Accessibility for Deaf Signers

In this post, we summarize the findings of our research paper Social App Accessibility for Deaf Signers, which was accepted to the ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW) 2020.

People use social apps to connect and share with one another. However, most of these apps are not designed with the needs of people who are disabled or Deaf in mind.

We interviewed and surveyed d/Deaf signers — that is, d/Deaf¹ or hard of hearing people who use a sign language, like American Sign Language (ASL) — to better understand their experiences and provide suggestions to improve accessibility inclusivity of these apps. Therefore, we asked participants about how they use social apps and the barriers that they face in doing so.

Many people who are Deaf prefer communicating in ASL (or another sign language) over spoken and written languages like English. ASL is most different from common languages used on social apps, like English, in its visual nature. Relying entirely on visuals with hand movement, shape, position, orientation, and facial expressions, ASL does not yet have a widely accepted written form. Therefore, sharing ASL on apps with static and text-based content can be more tedious or challenging than sharing in written languages.

Social apps can be more accessible for Deaf signers

We surveyed 60 d/Deaf signers to learn about how they currently communicate with social apps and the challenges they face in doing so. The survey questionnaire was based on interviews we conducted with seven d/Deaf signers. As one might expect, captioning audio content was the most common accessibility barrier, mentioned by every interviewee and 77% of survey respondents. Though automatic captioning tools are improving, they are not of high enough quality or widespread enough to act as a solution to this issue. Platforms and content generators need higher-quality, scalable, easy-to-use captioning tools.

Comparing survey respondent language preference in daily life versus use on public feeds in social apps.

Paradox: preference for visual but use of text-based communication

Our participants showed strong preferences for signed and other visual forms of communication. Specifically, 74% of participants preferred communicating in ASL, or ASL and English equally. Interviewees expressed enjoyment in using things like emojis and GIFs, as one person stated:

“OH yes [I like sharing GIFs and emojis]… Because we use our facial expressions all the time for ASL.”

At the same time, written English was used by the most participants and the most frequently on public feeds. 77% of survey respondents who shared on public feeds used written English and 51% used ASL. People explained that using written English was easy, fast, and understood by d/Deaf and hearing people alike.

Why aren’t people using ASL on social apps?

Leveraging findings from our interviews, we created a list of nine barriers to sharing videos with ASL on social apps, the most prevalent of which we grouped into three categories:

• ASL content is not easy to create. ASL uses both hands, and signs can easily range from the top of the head to the waist. As facial expressions are integral to the language, one must be well-lit to be understood. Some of the issues that participants reported included a small camera view that cuts off much of the signing space (72%), difficulty in propping up the phone to sign with two hands (70%), and ensuring the signer is well lit (65%).
• ASL content is not fast to create. 77% of our respondents stated that the process of recording and uploading a video takes too long, and the issues related to difficulty in physically recording sign language likely increase this time. Relatedly, 72% of respondents indicated that uploading videos requires too much data.
• ASL content could not be consumed right away by d/Deaf and hearing people alike. Surprisingly, the barrier that affected the most participants (and had the strongest effect) was the difficulty in creating captions of signed content for hearing friends (89%). There are currently no tools that automatically caption ASL or even sync English transcripts with signed content. This indicates that Deaf signers often need to choose between enduring a lengthy, tedious captioning process, or sharing without captions, potentially excluding large numbers of non-signers.

How can we do better?

We gear our suggestions to researchers and social media app providers.

Researchers can:

• Improve captioning technology for both ASL and spoken English. This involves building ASL recognition technology to allow for automatically generated or AI-assisted captions.
• Investigate hardware solutions to improve the ease of recording sign language (e.g., hands-free on-the-go video recording solutions, wide-angle cameras, lighting the user when they use the front-facing camera).

Platforms can:

• Improve content captioning. There are several ways to approach this issue, including hiring human captioners to caption the most popular or important content, or allowing and encouraging crowd captioning (e.g., via hosting microvolunteering days, awarding badges to those who caption content, etc.)
• Support other forms of visual ASL communication. For instance, Deaf artist Jessica Flores created a set of animated stickers that allow for fast and fun communication of common ASL phrases.

We hope that this work spurs initiatives towards making social apps more supportive of Deaf signers and their communication practices. We invite you to read our paper for more details about communication patterns and accessibility barriers faced by Deaf signers on social apps.

[1] Lowercase “d” deaf refers to people who are deaf in terms of audiological ability to hear while capital “D” Deaf shows identification with Deaf culture. It is important to note that ASL is not just signed English; it is a complete, distinct language with its own vocabulary and grammar structure.