Beyond Audio Description: Exploring 360° Video Accessibility with Blind and Low Vision Users Through Collaborative Creation

4 min readOct 29, 2023

In this post, we present and summarize the findings of our research paper, Beyond Audio Description: Exploring 360° Video Accessibility with Blind and Low Vision Users Through Collaborative Creation, which was published at the ACM Conference on Accessible Computing (ASSETS) 2023. This paper was authored by Lucy Jiang, Mahika Phutane, and Shiri Azenkot.

Despite being touted as “immersive” and “interactive,” 360° videos are largely inaccessible to blind and low vision (BLV) users. 360° videos capture a panoramic field of view and allow users to engage with different parts of a scene depending on where they orient their bodies or turn their heads. Typically, videos are made accessible through adding audio description (AD), which is the additional narration of visual information on screen.

However, there is limited guidance on how to apply AD to 360° videos without compromising their immersive quality. Prior works have only focused on audio descriptions, but what about other ways of thinking about video accessibility? Our research explores ways to enhance 360° video accessibility while conveying feelings of immersion and engagement in a nonvisual way.

To understand how to make 360° videos more accessible, we conducted a two-part study involving (1) individual interviews and (2) collaborative design workshops. While our interviews helped us understand participants’ prior experiences and individual preferences, the workshops allowed us to synthesize perspectives and co-create accessible 360° video prototypes with participants. Of our 14 participants, nine were BLV AD users and five were sighted AD creators. To ensure a wide range of perspectives, five BLV participants were professionals in the AD industry and had intersecting experiences as both users and creators of AD.

During the interviews, we asked participants about their experiences with AD consumption or creation, inquired about their experiences with 360° videos, and conducted a brainstorming session using a video probe. The workshops included a design activity where participants collaboratively designed an accessible video experience for a 360° video based on the Super Mario Brothers franchise.

We summarize the findings from our study below.

Findings

Our findings uncover key design considerations for enhancing the accessibility and immersion of 360° videos for BLV viewers, which range from the linguistic presentation of video descriptions to multisensory interactions and feedback. Check out our video prototype of one of the scripts created during Design Workshop 1!

Linguistic Preferences

We found that the linguistic presentation of audio descriptions can greatly impact a user’s engagement and immersion. Almost all participants preferred audio descriptions that used first- or second-person pronouns (such as “I” and “you”) as they found them to be more personal and immersive, compared to the third-person pronouns (such as “they”) used in traditional AD. Participants in Design Workshop 1 also explored a technique where characters could serve as narrators, which built descriptive narration into a character’s dialogue.

Aural Preferences & Sound Design

Additionally, the quality and spatialization of audio strongly impacted BLV users’ experiences. Participants had diverse preferences for the spatial location of AD. Most preferred the “tracked” option for description, which placed the descriptions in the direction of action in 360° scenes, as they found it to be the most immersive and interactive. However, others enjoyed the “omniscient” or “friend on sofa” options for increased clarity and accessibility. These findings underscore the importance of flexibility and personalization in creating immersive environments for BLV users.

Participants also mentioned how audio cues could support their agency in 360° video experiences. They preferred subtle cues, such as beeps or beacons, that could guide them throughout the space without interfering with their independent exploration.

On the left, there is a screenshot of the video probe featuring Luigi plunging a bathtub drain. On the right, we have the participant-created description, reading “As Mario, we watch Luigi, a tall thin plumber with a large mustache in a green hat, in a tub plunging a bathtub drain” and the audio cues, reading “Suction cup plunging and spatialized AD.”

Multisensory Interactions

All participants were also interested in novel ways of interacting with videos in multisensory ways, which included haptic feedback and tactile augmentations. Participants’ proposed ideas were inspired by theater pre-shows, museum exhibits, and video game controllers. They also noted that haptic and tactile feedback could provide additional access for deafblind users. For other senses, some participants thought smell and taste were interesting, comparing the experience to being at a 4D theater. However, others had more reservations about grotesque smells or tastes making them nauseous.

Including BLV AD Creators

Part of our study focused on understanding whether and how BLV AD creators contributed to video accessibility efforts. During the workshops, blind and low vision AD creators and users contributed unique insights. These included detailed questions about characters and discussions about the inaccessibility of existing media which were grounded in their lived experiences. Furthermore, participants acknowledged that AD creation is a collaborative effort, a finding corroborated by the group interactions observed during our design workshops.

Discussion & Contributions

Our findings point us to future directions to make media more accessible. Our insights on holistic accessibility can be generalized across many types of media, including traditional videos, images, video games, and extended reality. Additionally, prior works have taken a utilitarian approach to media accessibility, without considering why BLV people engage with content — for entertainment and enjoyment. We recommend future work to keep in mind that BLV audiences deserve to have access to both usable and enjoyable video experiences, and to further explore creative methods of providing access that go beyond traditional audio descriptions.

Ultimately, we hope that our work sparks conversation and inspires future efforts — both in academia and in practice — to make videos and other media more accessible, immersive, and enjoyable.

We invite you to join us on this exciting journey of making immersive media more accessible! Please read our full paper and reach out (lucjia@cs.cornell.edu) if you have any questions.