Finding Love
Finding Love is a fully interactive, Virtual Reality story that transforms emotions into art. Moving through five distinct chapters, you experience the journey purely through visuals and sound.
Created as an exploration in conveying feelings through visuals, we learned that there is a large audience that appreciate this form of art.
In the fall of 2016 we began thinking about a new experiment to explore how a story could be told primarily in VR but still be impactful in other forms such as a desktop and mobile website.
Storytelling in VR
While we began experimenting in VR earlier in the year, we decided to create Finding Love as a full project to be released and working on a wide range of devices.
Starting with the idea of telling a love story through code, we concepted five chapters with their own unique visual style and feelings. We wanted the user to be able to understand what is happening in the story with sight and sound; no words.
Each chapter’s interactions were designed to work not only in VR but also on a traditional desktop, tablet or mobile screen. For example, in the first chapter, the physics simulation is a fun interaction on a touch screen but elevates to a new level when it becomes a physical presence in VR.
While the primary goal of the project is to tell a story in VR, the secondary goal is to make it accessible for everyone via the web.
In the story, the user’s avatar is a gem that is paired with another gem of a different color and shape. A unique interaction in VR is the user being able to reach out and take hold of their gem to begin the story.
Connecting People in VR
At the end of the experience, the user learns that the other gem was actually another user sharing the experience at the same time and their approximate location is revealed.
Our desire in this project, as well as many others, is to connect people in simple ways that make a meaningful impression. The other gem’s movements in the experience are directly impacted by the partner’s input whether desktop, mobile, or VR.
Web technology, specifically WebSockets and WebRTC, provide a solid foundation to create connected experiences with technology that is already enabled on billions of devices.
WebVR
WebVR is an emerging technology that we believe is going to fundamentally change how VR content is experienced and bring it to mass audiences. With companies such as Google, Facebook/Oculus, Microsoft, Samsung, and more pledging support to build VR browsers we will see WebVR become accessible to the mainstream in 2017 and beyond.
From a user’s point of view inside a VR experience there is no difference between VR via the web and VR downloaded as an app. Where WebVR is interesting is the way you get users into the experience.
Currently, a VR experience is downloaded through an app store via a large download, typically hundreds of megabytes or more, as well as a powerful machine.
With WebVR, an experience is loaded simply by visiting a URL where the content is loaded like any other website. Finding Love, for example, is about 1.5 megabytes and loads very quickly.
WebVR will run on any platform so code written to target the web will work on Oculus Rift, HTC Vive, Google Daydream, Cardboard, along with forward compatibility with new devices that come out in the future.
Platforms and Devices
We created two separate paths towards the goal of making Finding Love accessible to a large audience; first optimizing for multiple VR platforms and second optimizing for a large range of devices.
The VR landscape as of the Finding Love launch is HTC Vive and Oculus Rift are the high end hardware, requiring as specialized machine and a substantial head-mounted-display.
On the low end, Google Cardboard and similar headsets provide lightweight lenses to view content.
An emerging middle ground, likely to become the dominant use of VR, is high quality mobile VR such as Google Daydream and Samsung Gear.
While Finding Love is an amazing experience on Oculus and Vive that puts you directly inside the story with the freedom to walk around, it also runs in a Cardboard headset on any mobile device.
For non-VR, we use the same techniques we’ve developed for client projects that enable us to detect device hardware capabilities and make decisions at runtime about features ranging from number of particles to post-processing visual effects.
A Google Pixel or iPhone 7 will receive all the available effects, but an iPhone 5 will see reduced lighting effects and less particles in order to run smoothly at the maximum 60fps framerate. You can learn more about our techniques in the Find Your Patronus case study.
Technology & Visual Effects
Finding Love is the latest evolution of our tools and workflow and attempts to push everything just a bit further.
It makes for first use of a new internal library we’ve developed to make multi-user gameplay over WebRTC extremely simple to implement in a project.
When a user begins their experience they are matched to another user over a WebSocket connection to a matching server. When a match is made, the two users connect directly to each other over WebRTC and exchange information such as location and input data in order to see the other user’s motion reflected in the partner gem.
For post-processing, we created a depth of field effect by taking advantage of our “FXLayer” toolset that makes it simple to render the scene with different shaders. In this case, rendering a depth pass allows us to selectively blur different parts of the screen to create a convincing effect.
We also love to come up with simple tricks to create visually interesting effects that are simple to render. We created a refraction effect on the gems by taking the previously rendered frame and passing it into the gem’s forward render shader and sampling it based on screen-space UV coordinates and offsetting the lookup with a wavy normal map.
Audio and Tech by Plan8
The music has a big part in telling a story, especially a story without words like Finding Love. Being a linear experience, we could work with our composition much like we would when scoring a film, creating the music to follow the story without having to wait for user interactions and have the music looping.
We composed each part of the soundtrack to catch the feeling of it’s chapter. So each chapter has it’s own instrumentation, tempo and chord progression, with transitions in between.
To create a contrast to the clean, digital world of Finding Love, we wanted the music to be a mix of digital and organic sounds. We ended up using mostly analog synth sounds, often with some noise and pitch modulation to make it sound a bit off.
Spatial Music
Since it’s a VR experience we wanted to try and see if we could make the music feel more integrated and present by using audio spatialization.
We did some experimenting and ended up using three sound sources placed in a triangle around you in combination with one normal stereo mix. The sources placed around your head follow your movement in room scale VR but are not head tracked, so you’ll hear some sounds panning around when moving your head. Some instruments like drums, bass and big synth pads did not work well positioned, so we kept those in the stereo mix. It’s a really subtle effect but it adds that extra feeling of presence.
We also attached a sound source to your gem, so if you have a game pad you can move around the sounds coming from your gem. We’re using the Web Audio API and it’s panner node with the HRTF setting, which is really powerful for creating the illusion that a sound is moving around you in 3D space.
Viewing in VR¹
If you have an HTC Vive or Oculus Rift you can experience the story by visiting webvr.info and following the install instructions. From there, simply visit findinglove.activetheory.net.
If you have a mobile VR headset such as Google Cardboard visit findinglove.activetheory.net and select the “Cardboard” button.
If you are looking to test Daydream, a rough beta version can be accessed using Android Chrome Canary or the Chromium build at webvr.info.
WebVR is still extremely cutting edge and is the spec is still changing. We will be using this project as a testbed and will be updating it throughout 2017 with support for new VR browsers.
