How to turn flat videos into VR experiences

This is a quick summary of how we turned flat drone footage into an immersive WebVR Experience.

Published in

Hinderling Volkart

4 min readOct 24, 2019

Outtake from https://www.myswitzerland.com/en-ch/ VR experience

In our latest project for Swiss Tourism we got to work with some amazing drone footage showcasing the most beautiful places all around Switzerland. Alongside the desktop version, we created a WebVR mode that enables users to experience these videos in VR using a customized interface to switch between locations. WebVR allowed us to target all kinds of VR devices at once without having to build one version per device.

The main challenge: Mapping Flat Videos onto custom Geometries

VR devices allow for a 360 degrees look-around. Unfortunately and in our case, the drone footage was not captured in 360 degrees but flat instead. To enable a VR mode for these clips we had to come up with a solution. Thus, we started to build custom 3D geometries and mapped the video onto them. These geometries had to comply with several criteria. The main one was to prevent distortions as much as possible. At the same time, we wanted to provide users with an immersive feeling by allowing them to look around and explore landscapes. We implemented a basic sphere shape geometry. Based on that, we carried on to manually adjust it until we had found a grid that suited our needs and catered for a 16:9 aspect ratio.

Camera Position and information overlays

In conjunction with a suitable geometry that could present a video in a VR space, the position of the camera played a vital role. Depending on the position of the camera and the distance of an object in the video, the amount of overlayed information had to adjust and vary. Finding a sweet spot and thus achieving a satisfying result came down to a lot of tinkering. Constant testing using a VR-Headset all along the process allowed us to iterate fast.

Hard Edges

Once we had achieved a satisfying look for our videos, another visually unappealing thing stood out. The video looked framed. When turning one’s field of view towards an edge of the video, the transition from video to the background created a hard cut. To avoid it, we added a subtle alpha mask that fades the corners of the video geometry. By then using a sky sphere in the background and building a dynamic gradient based on the color value taken from the top and bottom of the video, we achieved a nice and soft fade-out from video to background.

Smooth crossing between Video and Dynamic Sky Sphere

UI elements

While turning a flat into a VR capable video was one challenge, allowing users to navigate through 48 clips and obtain facts and figures about points of interest was another one. We created three different options to navigate. A «skipper» to go forward and backwards, a map allowing to pick a location and a calendar to swipe through. The corresponding UI elements appear in the foreground of the video. We added spacing to elevate them and thus separate foreground and background. To find an optimal position, we applied the same technique as we used when adjusting the camera by directly positioning the UI in VR.

To trigger these menu options we decided to only show them when users look down. Applying this principle allows little to no distraction from the main experience. Looking left and right triggers the corresponding «skippers».

A-Frame Community

We built this experience using A-Frame. A-Frame is a relatively young framework. The fact that it’s constantly under development often raised questions when things didn’t work the way we would expect them to work. Luckily, the A-Frame community was very supportive.

Here are some references and entry points to the community:

Slack: aframevr.slack.com
Stackoverlflow: https://stackoverflow.com/questions/tagged/aframe
Supermedium Discord: https://discord.gg/tGYjkYr
Blog: https://aframe.io/blog/
Subreddit: https://www.reddit.com/r/WebVR/

One last thing

To set a titles’ typo we deconstructed words in space. By picking up a motion similar to the video, the font gets dismantled and reassembled while flying letters surround the user. More about that soon.