Ending the Hype Cycle: Today’s Challenges and Opportunities in Improving the 360 VR Video Experience
VR has been in a lasting hype cycle since 2015. Until now the focus on VR, and the tremendous potential this technology holds, has been on gimmicks and wizardry — flashy, conceptual videos and the announcement of products that weren’t yet market-ready. However, this year marks a shift in focus toward VR’s practical uses and how to deliver on the promises made to consumers, especially with 360 VR videos. We’ve already seen evidence of this trend at recent industry events including CES , MWC, and GDC/VRDC 2017.
Among the many VR formats, 360 VR video has stood out as a lower-hanging fruit opportunity than others. But what the industry can currently deliver in 360 VR videos is still far behind what our eyes are looking for. A common complaint about many existing 360 videos is that they appear blurry and choppy. For many consumers, it is frustrating and confusing.
An In-Depth Look at VR Video’s Reality
Why do VR videos appear blurrier than our regular videos today?
Human eyes of 20/20 vision see about 60 pixels per degree, making 16K the ideal resolution for 360 degree videos. Most modern smartphones, if viewed from an arm’s distance, or TVs, if viewed from a few feet away, can deliver more than 60 pixels per degree thanks to many years of advances in display technology. As a result, they’re both capable of delivering a pleasant experience to 20/20 eyes.
However, VR requires much higher resolution due to its extremely large field of view — a complete sphere. To achieve the same pixel density in VR, you need about six to ten times more total pixels. Therefore, the industry considers 4K as the minimal resolution requirement for VR videos.
Of course 4K is just a starting point. Just as regular video evolved from 240p in the 1980s to 1080p (and now moving toward 4K), users will also expect a VR to evolve.
A better VR experience will require improvement in three key areas:
- Higher resolution from 4K to 8K, 12K and eventually 16K to achieve 20/20
- A shift from monoscopic to stereoscopic to deliver 3D effects
- Going from 360 videos to volumetric videos or even light field to achieve full freedom in VR!
To achieve progress in any of these three aspects, ten to hundreds times more pixels will need to be created and delivered.
If 4K is today’s ‘must have’ standard and 8K capabilities are in existence, why can’t general consumers already watch VR videos in 4K?
This is because delivering a VR video experience has a long pipeline. To deliver a positive experience, it requires that every step in the pipeline be done right and up to the current industry standard — from creation to streaming to display and playback. Although nowadays all professional level VR cameras can already shoot at 4K+ resolution, and 360 videos and VR headsets can also display 4–6K resolution, the crucial step of VR streaming is not able to support the 4K standard yet. In the US, according to Akamai’s 2016 Q3 report, only 40% of home wifi can support streaming 4K monoscopic video, and less than 10% of home wifi can support streaming 4K stereoscopic or 6K monoscopic.
Outlook on VR videos’ future
Although VR video experiences are not up to the industry’s expectations or consumers’ minimum standard yet, its short-term and long-term future is quite promising. Today, technological advancements across all three major limitation areas — creation, streaming, and display — are developing to enable creators to make better VR video experiences.
On the content creation side, people are moving to bring higher resolution, 3D effects and interaction into VR videos. Starting in 2017, 4K has become a standard for both VOD and live streaming, and at the SXSW 2017 conference taking place now, the VR production community is talking more about making 8K VR videos. GoPro Omni has been able to shoot and stitch 8K VR videos since mid-2016, and other cameras such as zCam and BlackMagic rig can shoot at 6K now. With the very first limited-scope prototypes of 8K VR live streaming and volumetric VR video production making an appearance at MWC 2017 and CES 2017 earlier this year, coupled with the continued improvement of video stitching and editing workflows, the evolution on the content creation side is well on its way.
On the display side, new headsets with significantly improved resolution are hitting the market. For example, at VRDC 2017, LG announced their new desktop VR headset that has a resolution of 1280 x 1440 for each eye, which is higher than HTC Vive and Oculus Rift and capable of delivering up to 6K clarity. On the mobile VR side, Sony was the first one to deliver a 4K mobile phone, and it’s expected that an 8K resolution capable mobile VR device will be available in the near future.
Moreover, standalone mobile VR headsets are also emerging. Qualcomm recently demonstrated a reference design of their stand-alone VR headset at VRDC and said they are aiming to launch it commercially in the second half of 2017. Facebook, Intel, AMD, and Samsung are all also exploring stand-alone headsets, which will allow positional tracking and free movement within mobile VR.
On the content delivery part, we see two kinds of efforts moving forward.
First, we see the telecom industry speeding up their pace to bring 5G networks live, fundamentally improving our data transfer capacity. At MWC 2017, not only did Intel partner with Ericsson to show a 5G network prototype, almost all the major players had something to say about 5G. 5G could potentially achieve commercial use in 2020, and by 2025 around 1/8 of all mobile phones will be able to get a 5G connection. However, compared to the pace of user experience upgrades, this might not be fast enough.
Secondly, since 2015, a new technology in the VR industry called foveated rendering and streaming has emerged as a more efficient streaming solution, solving the bandwidth issue and filling the VR streaming need gap. This technology works by only streaming the field of view area that VR users see at the time. Whenever a user’s head or eyes move, the view follows and the streamed data adapts accordingly. Several players in this space are taking two different technical approaches to deliver foveated streaming — some are viewpoint based, such as Facebook Dynamic Streaming, while others take a tile-based approach, like Visbit View-Optimized Streaming. Although this new technology is still in the experimental stage, VR industry leaders from Oculus, Google, and Vive in attendance at VRDC 2017 in San Francisco all mentioned that foveated rendering and streaming, thought difficult to do correctly, is the inevitable way to go.
Conclusion
Consumer needs for a better viewing experience will drive advances in streaming for the VR industry. The majority of users, sadly, are still watching 360 degree videos in 1080p. But technological breakthroughs are being made, even as this is being read. We foresee that 4K 360 degree videos will become the standard for the industry in 2017, and a better quality and more immersive and interactive VR video experience will be a reality in just a few short years.
This post can also be found at https://virtualrealityreporter.com/ending-hype-cycle-discussion-todays-challenges-opportunities-360-vr-vide/
