Building a Deaf VR Experience

Simple Misunderstanding at Latitude Festival 2017

My proposal to create a virtual reality installation about deafness was the basis of my successful application to the 2016 Mead Fellowship award from the University of Arts London. I wanted to tackle a subject that was close and personal to me and there is nothing more personal than my deafness.

I was born profoundly deaf and I have had a cochlear implant since I was 2 years old so I’ve never known otherwise. Over time I’ve become more aware to understand the impact deafness has had on my life growing up in mainstream society, from not being able to hear public speakers clearly to needing subtitles on the TV. The cochlear implant is a fascinating piece of bionic hardware that still amazes me. However, as wonderful the cochlear implant has been in helping me access sounds, it’s never going to reach the level of natural hearing.

Simple things in day-to-day lives that hearing people take for granted can have a huge impact on deaf people physically and emotionally. I’ve occasionally struggled to follow spoken group conversations and to hear properly in noisy environments.

My aim for the award was to introduce hearing people of all ages to these difficulties that deaf people face on a daily basis. The biggest challenge I faced was how?

What is it like to be deaf?

A cochlear implant

I asked myself this question and put down a list of things detailing what it is like being deaf. The list ended up with a variety of ideas, words and experiences both positive and negative:

· Deaf culture — One thing I am proud of the most about being deaf is being part of a wonderful culture. I’ve met many people from the UK and internationally from all walks of life and there is a shared belief that being deaf is something we all appreciate as it makes us unique.

· Sign language

· Vibrating alarms

· Subtitles

· Missing trains — I’ve had an issue with public announcements over speakers, fortunately these days we have a lot of visual information but sometimes it goes wrong or isn’t enough.

· Group conversations — these have a huge impact on me. I always find it hard to follow people talking and I become an outsider in these situations.

· Chatting to people in bars — there have been many, if not always, times where I’ve just nodded my head when I’ve been talking to someone in a noisy bar and I can’t hear them well. Sometimes this can lead to humorous misunderstandings.

I have a group of friends who are also deaf so I shared my thoughts with them and saw what they came up with. The most common themes were similar to mine although I found that my friends mainly focused on the negative parts, parts which could be improved with better awareness and technology.

Once I had all these notes together it felt natural to use the VR installation as a means of raising awareness for those situations where a deaf person would struggle, such as the need to lip-read, being in a noisy environment and being in a group conversation.

Designing for VR

Simple Misunderstanding at Secret Garden Party 2017

The next step was to translate these ideas into a VR enviroment. Here’s where the fun and creativity really started.

My software of choice was Unity as I have had past experience using it to develop games. I did some research into Unity and VR and it ticked all the boxes for what I was aiming to create. I had the HTC Vive so I was looking for SteamVR built in from the start. It felt comfortable to use an engine I was familiar with yet learn something new and develop new skills for this project.

I took several of the ideas that I had with the aim of making a 5 minute experience. In the end after many weeks of testing and changing the narrative structure of the installation I settled on 3 main issues to address including an introduction and ending. For people who use VR for the very first time it’s important to give them time to settle into the scene so that they become comfortable with the headset and the virtual environment they’re in. Then they will focus more on the content and narrative as opposed to the novelty of using VR for the first time.

The scenes I settled on were:

· Talking to a person individually, needing to face them and follow them to hear them clearly

· Being in a group of people struggling to follow the conversation

· Silence, no hearing at all

For each scene I wanted to recreate the feelings and emotions that deaf people can experience when they struggle to hear, but to also suggest a way in how it could be improved. Hopefully educating the user on how to approach deaf people in future. The ending scene then summaries all the experiences and provides the user with tips and facts about deafness.

Each scene takes advantage of the VR environment. The colours of a room change depending where you’re looking and listening, the sound quality depends on who you’re facing and there’s a scene where the room literally starts spinning making you feel uneasy.

I was inspired by the minimalist set design approach in theatre where props are used to indicate a setting. I chose to have a scaled down environment where the focus is on a person or their face and the environment is secondary. In some cases the environment takes over as it can when a deaf person is in a noisy environment which was fun to experiment with.

The project wasn’t aimed to get sympathy from people but rather get an empathetic response based on a new understanding of the problems deaf people face.

I will cover more about how I created one of the scenes in detail in the next few weeks.