How to deal with audible inner thoughts in VR
Results and conclusions from audience research of 360 VR film ‘Anyways’
Inner Thoughts
Being able to see the world through the eyes of a character in VR may sound familiar to you. But hearing the inner thoughts of your character is not as often done yet as we’d thought by now. And why not? It is a very unique was of using VR. Hearing inner thoughts from the inside of a character makes use of the medium in a way that cannot be accomplished with any other medium. But because using VR this way is quite new, it doesn’t always work out the way you plan yet. There are so much things to take in account, on top of all the other things that are new when making VR. How do we find the balance between the amount of spoken dialogue and the amount of inner thoughts heard on top of that?
In normal life we think also while others are speaking, so why not hear dialogue and inner thoughts simultaneously in vr? Would the inner thoughts distract from the story told? Do people mind not being able to follow everything around them when this happens? These and other more practical questions were the base of our audience research. In this blog we will tell you what it is we researched and what our findings and conclusions are.
Aftertalks
But first, a little background information for our audience research, concerning overall audience reactions on Anyways. We have showcased Anyways in various festivals and cultural platforms in the Netherlands and abroad. Numerous audience members have provided useful (but at times incompatible) feedback during the after-talks.
In general, many people are speechless after a viewing; they cannot find their words to express themselves. This is, in our opinion, due to the innovative way of storytelling-via a first person perspective view while hearing a conversation with added inner thoughts. The majority –but not all– of the participants seemed to understand that they hear the inner thoughts of their character through the bone-conducting headphone and that this is different than the text of the general conversation. The majority –but not all– audience members experienced an overload of information.
For most, the overload is caused by the layeredness between inner thoughts, simultaneous conversations and interactions between the characters, visual image experienced by a first-person perspective, and additional haptic stimuli. Others needed more time to look around in the VR environment and to position themselves in the train wagon prior to any conversation. Most participants find pleasure in getting ‘into the head’ of their character by means of the experience of the inner thoughts.
Most people in the role of the Swedish DJ Lars reported to feel intimidated by the constant attention of the other characters. Some people found it alarming to be touched by a stranger (in this case the hostess), especially when they are unable to see the one touching. Yet, most people express that external touch enhances the feeling of embodiment and leads to a more realistic experience of the situation. The dislike towards one’s character led to feeling distracted from the story for some participants. Others found it very interesting to be in the head of someone so different from their own character.
Audience Research
These findings have caused us to start asking some questions that never got their full answer, because the participants differed so much in their reactions to the experience. To find these answered we set up a more controlled setting of Anyways, where only two people at the same time got to experience the story; one man and one woman. They each only got to experience the story from the perspective of their own sex, so one male and one female character. Overall the participants did not have prior knowledge that Anyways includes inner thoughts. We had two afternoons where we did this study, one at the Filmacademy and one at the Theatre Academy in Amsterdam.
A total of 24 students participated in the research. The participants were students from different studies; ranging from different types of dance, to acting, mime, film production, sound design and visual effects. Their level of experience with VR also differed; three students had never experienced VR before, 12 students had only experienced VR gaming before, 14 of them had already experienced other story-based VR and four of them even had made VR themselves already. After the experience ended, they only got the chance to speak to each other after we’d asked them our questions by interview. Some of those questions they could answer with a number ranging from 1 to 10 and some were open questions.
Here is a list of our questions to the participants and what could be concluded from their answers:
How clear was it that you experienced the story from the perspective of your character?
This apparently was very clear, which is a good base of knowledge when looking at the other data of questions particularly about inner thoughts.
To what extent did the inner thoughts help you understand that you were that character?
Although overall still positive, the answers to this question were a bit more scattered. The reason for this is unclear, but it could be due to the fact that some participants might have been aware of other factors for this as well. In Anyways there is also the experience of a body, that might have a big role in embodiment of the character. Also being spoken to directly by other characters could have a role in this. Some participants might have been aware of this and others might not have been.
To what extent did the inner thoughts help you feel present in the story?
The result to this question is quite clear. The inner thoughts helped almost everyone to feel more present in the story, although some more than others.
How clear was the difference between the spoken words and the inner thoughts of your character?
The difference between spoken word and inner thoughts were clear for most participants, but not for everyone. What is interesting though, is that when looking at the answers of the students that gave a score of 6 or lower, all except one of these students came from the Film Academy. This same question was asked in a former study with a different VR experience (Out of Sight) that used inner thoughts. This study was done with participants of a more wide range of ages. The answers to this same question also got quite high numbers, except from four of the participants. These contained the only two filmmakers that entered that study, an animator and one other person.
Both Anyways and Out of Sight used different ways of making the inner thoughts audible, but both experiences did this in a very different way than is done in film. In film the thought of a character is made clear by putting reverb on the speech. Anyways uses bone conduction headphones to let participants hear inner thoughts and little speakers positioned in front of each participants mouth to make the dialogue audible and spatial. Out of Sight let everything hear on headphones. They took away all reverb on inner thoughts, thereby making a clear difference between the inner thoughts and the spatial sound ‘outside’ of the character. It is possible that hearing any of these new conventions used in VR might be confusing for filmmakers, who are used to work with a very different set of conventions.
How clear was it that you were the only one who could hear the inner thoughts of your character?
This was also very clear for the participants. The only ones who filled in a 2 or a 3, were students who had never experienced VR before.
At which moment did it become clear that you could hear the inner thoughts of your character?
Most of the participants found out during the first moments of the experience that they were able to hear the inner thoughts of their character. Three of them only found this out after the experience ended. The findings from the earlier study with Out of Sight had similar results. We hope that as people get more used to the idea of hearing inner thoughts in VR, they will start to recognize inner thoughts earlier into the experience.
To what extent did you agree with the inner thoughts of your character in Anyways?
The answers to this question by itself was not relevant to our research. It was only asked to compare the answers to that of the next question. That comparison being why we asked this question.
To what extent would you normally have agreed with these thoughts?
The answers to this question by itself was not relevant to our research. It was only asked to compare the answers to that of the previous question. That comparison being why we asked this question. What we found was that there was hardly any difference between the answers. Meaning, experiencing VR with inner thoughts overall did not make the participants agree more or less with the character whose thoughts they got to hear. However, some of them did express later in conversation, that when they already agreed a lot to what the character was thinking, this helped them in connecting with the character. It made them more present in the story and helped them embody the fictional character. This could be an interesting fact to check in a next study.
To what extent could you follow the storylines between the characters?
This is the part where Anyways has always received the most comments on. There are six characters in the train talking to each other, resulting for viewers to be able to hear multiple conversations at the same time. On top of that you sometimes also hear the inner voice of your own character. This makes it impossible to hear everything that is going on. A viewer is forced to make choices in where to listen to. Interestingly enough, over one third of the people responded to this question with a seven or higher. Meaning they still had the feeling of being able to follow most of the story going on.
To what extent were the inner thoughts distracting from the story?
The answers to this question show that the inner thoughts played a big role in distracting from following all storylines.
If you felt that the inner thoughts were distracting, can you explain why you felt this?
Some of the answers to this question related to technical aspects, like sound levels not being high enough:
“Because I didn’t understand the meaning of the text. And because I couldn’t really hear the inner thought properly due to the level of the sounds.”
Sometimes it was due to needing some time to understand the concept of inner thoughts, before they could focus on the story itself:
“I was searching for the difference between speech and inner thoughts and that was distracting me, it took me out of the experience. It was mostly in the beginnen, I needed some time to adjust to the new world.”
But most of the times, people explain that it was just hard to know where to focus:
“sometimes there were things happening outside of my character while she was thinking and then I did not know where to focus.”
If any, what other reasons prevented you from following all the storylines between the characters?
Five of the participants gave no other reasons. Other participants named some technical reasons, like sound levels, the sound not being synch, being able to hear the bone conduction headphone of their neighbor or not being able to find the subtitles at times. But the most given answer to this question was that they had a hard time knowing where to focus, because there was too much going on at the same time. One comment that stood out, was from someone who said that the inner thoughts were actually helping him focus in this:
“There were too much conversations going on at the same time. The inner thought were actually helping, because they were about having that confusing about. Most of the times it helped me.”
Did it bother you not being able to follow all storylines? Why, or why not?
This question was added after we had looked at the results of the first day, so only half of the participants got to answer this question. There was no clear yes or no answer we got from these last 12 participants, as 58,3% said yes and 41,7% of them said no.
As to why someone would or would not be bothered by this, here are some answers of people who said they WERE bothered:
“I didn’t identify with my character so my mind was more interested in what the other characters were doing but i couldn’t focus on that.”
“I wanted to know what was happening and it wasn’t clear from the first moment on that not everything was important. Near the end I was less annoyed by it.”
“Because I couldn’t focus and I couldn’t hear and follow other conversations, so i was confused.”
And here are some answers of people who were NOT bothered:
“It puts me in the position to choose. I found out that not everything was important, so I had the choice to follow what I want. And the show will go on either way, no matter what I do.”
“I felt active and trying to be part of it. You need to be present for this. I enjoyed it.”
“I liked that you could switch from fully following to not fully following the storylines.”
It seems to be key to be able to surrender to the new format you are being put in. Some people can do this and other people can’t. This could mean they need to experience more VR like it, before they can. Or it could mean that when writing Anyways 2.0 Pips:lab’s scenarist could try to help the viewer more in where they should focus. It that is the case, it might help to think out a path of focus beforehand, that would be the most easy and logical to follow.
What was the added value of the inner thoughts for your experience of the character?
Almost all participants said there was added value. The value itself differed from being able to understand their character better, to better embodiment of the character (the feeling that you are the character) to the feeling of the character itself given more overall depth.
Do you have additional comments on inner thoughts?
Most of the participants used this part to repeat something they had said before. Two however had some interesting new comments:
“I got a bit paranoid in a way also, like people were able to hear my thoughts because I could hear the thoughts of somebody else.”
“it was very relatable and humoristic. it makes me aware that we have a lot of inner thoughts, this is normally not something I pay much attention to in life.”
This blog forms part of the Scenario and VR research trajectory, a collaboration between the Netherlands Film Academy in Amsterdam (AHK), Amsterdam University of Applied Sciences (HvA), and PIPS:lab, an Amsterdam-based collective creating multimedia installations, performances, and inventions. The research sprouts from the 360° VR movie Anyways (PIPS:lab, 2017) and includes audience research, design and development of two interactive scenario writing tools Dialogus and Paperol, two use cases regarding Paperol, and three workshops with scenario students of the Film Academy to test Dialogus. The blog series documents this research trajectory. The research is supported by RAAK.
