Beyond 360 Experiences

2016 will probably be forever known for the starry eyed excitement surrounding VR & 360 video. Even before this year started many in software development knew this year was going to be a watershed for the new tech. From the Oculus Rift’s consumer release, to the GearVR headset being bundled with Samsung’s S7 smartphone, and the continued promise of additional VR devices like Playstation VR and Vive. Hardware manufacturers from other parts of tech are quickly picking up the pace of VR, like GoPro’s Omni camera kit and even Sony announcing that there will be an upgraded Playstation 4, dubbed “Neo”, that will make up for the current console’s possible inability to handle the framerate consistency needed for smooth VR experiences. As seen from the GoPro hardware announcements as well as similar hardware now available in pro photography, 360/VR video creation is increasingly in demand. There’s hope in a number of tech industries that the consumer will continue this demand from this year and beyond.

Other than VR games, 360 movies captured by the aforementioned photography hardware are the next common use of VR tech. The default movies that can be downloaded for the Samsung GearVR are a great intro into VR and can give the user a sense of flying and feeling like they are really there at the locations. This is the current selling point for these experiences. However, 360 movies/videos do have their limitations, the first is their presented resolution. Unlike the VR gaming/360 rendered space, making 360 videos viewable in 1080p or greater in the VR space was/is still a bit of an issue and could take the user outside of the experience. Like most emerging tech, the most important task is to make sure it works at the most basic levels. Any kind of lag / framerate drop in VR will equal instant nausea for the user, so the ability for VR hardware to render 360 movies in full resolution wasn’t as important as simply being able to view a video smoothly. Capturing the content in high resolution originally held a challenge as well due to the rendering power needed to stitch essentially multiple videos together.

Both the hardware capturing the footage and displaying/rendering it are quickly resolving the resolution hurdle but one other issue still stands out with 360 videos…the fact that 360 videos are, well, not really interactive. Currently if you want to interact with a VR experience, it’s a game or a game-like rendered environment. 360 videos might be great now for those who’ve never wore a VR headset or used an accessory like Google’s Cardboard, but what are the next steps? Where do we take 360 video for those who want it to be more than a VR gimmick?

One of the easier steps for developers & producers is to make 360 videos like a “Choose-your own adventure” game. After a certain plot point in the video, having the user choose how they wish to progress will simply move them to another video that is based on that choice. This is something that can be a solution for 360 videos on the web, where mixing 360 video and interactive 3D live-rendered objects is not yet an option. This way of improving a static 360 video’s interactivity is still a bit static and predictable. The next logical step is to make them more like the VR games / game-like environments. Currently, for VR experiences it’s either you have a rather static 360 video or a pre-rendered environment, but not both. This is because 360 videos under the hood are usually 2D static textures in a single surrounding 360 sphere. When you see an object pass by you in a 360 video, it’s akin to watching objects move around a projection thrown on a wall, so interacting with them holds an extra challenge for developing/producing the content…in addition to adding more stress to the hardware rendering the scene. There also has to be some care in linking the spacal alignment of what you see in front of you that’s in the video versus a 3D object that technically in front of the “wall” showing the video content in the distance. With tools like Unity, Unreal Engine and others however, we can start to do this though. Much of why we don’t see this yet, despite maybe some hardware / web-based hardware limitations, is mainly since many people are simply not mixing the two, 360 Videos and game-like interactivity & environments. The door is mostly open for people to make these and more will be doing this over time as the veneer of 360 videos wear off if all consumers see is static / non interactive experiences.

When creating any kind of interactive application, always use gaming as the benchmark since all facets of entertainment and interactivity come from the gaming mentality space. Part of the reason why VR even exists is due to gaming and placing interactive, game-like storytelling is not only where VR gaming will go but also where 360 videos need to eventually reach as well.