Anyways: a social VR experience that enables to get into the minds of others
Anyways (created by PIPS:lab, 2016) is a 360° VR movie simultaneously experienced by six participants. They are immersed in a conversation between 6 eccentric personas sitting in a train compartment. Each participant experiences the dialogue from the point of view of one of the six characters, depending on which headset he or she has on. On top of this, the participant also gets to hear the inner voice of their character.
Vision
From the start, it was clear that Anyways would avoid a centralized story where the 6 characters experience the same storyline. Instead, we aimed to simulate a scene where you find yourself sitting in a train coupe with six chatty eccentric characters. Probably everyone has experienced such a situation where it is impossible to follow all conversations at the same time. The scene requires one to make conscious and unconscious choices where to channel their attention, which result in a partial experience of the story.
Different than in film in which you observe a character, in VR you embody a character. In Anyways, it was necessary to shoot the entire dialogue from POV to facilitate higher degree of immersion into the storyline. To personalize and heighten the experience of the participant, we decided to add a second layer of text; namely, the inner thoughts of each of the 6 personas. The inner thoughts offer each participant personalized additional information on the situation as experienced by the character they embody.
In Anyways, our starting points were:
- How can we create a social VR experience?
- How can we tell a single story experienced from multiple perspectives?
- How can we integrate the inner thoughts of each distinct character so that the participant can get inside the head of a character?
- How can we create a flow of interaction between the general conversation and the inner thoughts in the scenario?
- What kind of sound system would best suit the experience of a) inner thoughts and b) external dialogue?
- What elements are useful to borrow from existing (non-VR) film practice and to what extent do we need to invent new techniques and methodology?
Methodology
Telling a story from different perspectives with added inner thoughts suits the medium of VR; yet, it is a relatively new way of storytelling. To a large extent, in Anyways, we needed to invent our own methodology to fit our unique needs.
Omnidirectional sound system
Anyways aims to deliver a real life experience of a person sitting in a train compartment engaged in an external conversation and commenting on the others through inner thoughts. Hence two layers of audio are delivered: inner thoughts and the common conversation, which were recorded separately.
The general conversation can be heard through a tailor-made sound system affixed to the front-end of the VR headsets. Each headset is dedicated to one individual character, and the sound system delivers the inner thoughts of their respective character, as well as the background conversation. In the installation each participant is handed out a bone-conducted headphone to be positioned on the jawbone. The vibrating quality and sound texture of the bone-conducted headphone allows the inner thoughts to be experienced more intimately than the general conversation.
Design VR headset
In order to simulate a realistic impression of words coming from one’s mouth, we needed a design where we could position the sound speaker in front of the participants’ mouth (see red JBL boxes in the illustration). We also needed a design with a dystopian look. Musician and inventor Onyx Ashanti’s 3D printable design fit our needs perfectly; not to mention that it was well received. Most participants often ask for a picture with the masks on as souvenir. Who knew the Darth Vader look was making a comeback? (maybe it’s the splash of red!)
Blurring the virtual and the real
Blurring the boundaries between POR (plain old reality) and virtual reality has been a conscious strategy. This is done with the aim to enhance the immersion of the participants’ experience. Crossovers between the two worlds take place on various levels:
-Hostess: a performer wearing a specific costume welcomes participants and guides them into the experience. When the film starts, the participants see the same hostess as a hostess in the film. The first and last lines of the virtual hostess are simultaneously spoken out loud by the real hostess. The real hostess is present when the film ends. She re-welcomes participants and invites them to take a moment to recover from their trip.
-Sensory stimuli: Once the film starts, the hostess delivers cued external sensory stimuli specific to each character’s experience (the participants do not see this). Her actions include spraying a perfume and touching certain participants on the arm and legs. These actions correspond with the actions of the characters in the storyline and are executed at the exact time as the virtual contact and scents.
-Real reality: In the film the character Mehmet is seen wearing a VR headset by the other characters. The participant who is wearing the headset of Mehmet sees a totally different experience. At times, the participant gets to see other participants experiencing VR with dystopic headsets on. At other times, Mehmet dives into his own VR trip. A few of his adventures include flying over rail-tracks and entering an animation world.
-Self-reflexivity & medium recursion: Certain lines in the scenario indicate that the characters are aware of themselves being in a VR experience. “Where is this train going to? I mean virtually” said by the character Xray or “ I thought this would be a nice multimedia experience” said by Ray create a playful oscillation between heightened awareness of the mediated experience and full immersion for the experience of the participant.
After-talk
Anyways succeeds in creating a social VR experience on two levels. First, it brings six participants in a single storyline by means of its scenario that includes numerous interactions between the characters and its technical set-up. Second, Anyways is intended to be an after-talk movie. As explained above, in Anyways, the general conversation is biased and not told from a centralist point of view. On top of this, we have the personal inner thoughts of each character experienced by the participant…The story becomes complex! After the viewing, participants are encouraged to start a conversation to collectively figure out the plot they were in. What character they were, how they looked like but also what motivated their actions and why they reacted the way they did.
For most participants Anyways’ way of storytelling, in particular the inner thoughts, is new. We receive many positive reactions. At the same time, most participants cannot immediately find their words to describe their experience because they do not yet have the vocabulary or criteria to evaluate this type of experience. Others find it a pity that they could not follow all storylines because they were confined to the point of view and inner thoughts of their character. In order to receive systematic feedback, we decided to conduct an audience research [here is a hyperlink] to infer the experience of inner thoughts and the relationship of inner thoughts and the conversation among characters.
Credits
Concept: Keez Duyves
Direction & camera: Keez Duyves & Willem Weemhoff
Scenario: Willem van Weelden
Cast: Yorick van Wageningen, Sally Harmsen, Anil Jagdewsing, Thomas Boer, Yaron Mesika, Abner Preis, Zeynep Gündüz, Petra Ball
Audio recording & mixing: Daan van West & Tymen Bergman
360º sync player: Niels Johanson & Keez Duyves
Headset Mods & 3D printing: Onyx Ashanti
Additional fx: Nils Vaerno
Voice Over Promo: Pinar Karaaslan
Additional Optics: Joel Nesoptiek
Drone Shot: Vincent Lublink
Thanks to: Thijs de Wit, Yorick Heerkens, Andreas Scharfenberg, Lucia Kiel, Zola de Wit, Fred Rodrigues,Vava Stojadinovic, David Lammers , Yvo Sprey, Ohme Dirk, Hugo Reitsma, Aysegul Gündüz, Frank Hietbrink, Cinekid, Nina Juric & Letters are my Friends, de StormVogels, Orla C & Caspar ‘Europe Endless Express’ van Gemund.
This blog forms part of the Scenario and VR research trajectory, a collaboration between the Netherlands Film Academy in Amsterdam (AHK), Amsterdam University of Applied Sciences (HvA), and PIPS:lab, an Amsterdam-based collective creating multimedia installations, performances, and inventions. The research sprouts from the 360° VR movie Anyways (PIPS:lab, 2016) and includes audience research, design and development of two interactive scenario writing tools Dialogus and Paperol, two use cases regarding Paperol, and three workshops with scenario students of the Film Academy to test Dialogus. The blog series documents this research trajectory. The research is supported by RAAK.
