Making 12K 360º VR Streaming a Reality: Why and How We Did It
Today, 360 videos are mostly streamed at 1080p and occasionally at 4K. You rarely hear of 6K, and 8K is touted by some as perfect VR. However, some industry experts, such as Oculus’ Palmer Luckey, believe that 8K isn’t enough. Here at Visbit, we agree and we’re taking steps to go above and beyond that.
At IEEE Virtual Reality 2017, we announced our research project on 12K 360º VR streaming and zooming. We not only streamed a 12K 360º VR video over regular Wi-Fi and LTE, but also displayed it on the GearVR with the new zooming feature. You can watch the video of our CEO Dr. Changyin Zhou demoing this breakthrough, here:
In our demo, we created a 12K x 6K video, over which we laid a collection of smaller video clips at various resolutions. By double-tapping on the trackpad on the Gear VR, you can zoom into the smaller videos and view them without losing clarity. As demonstrated in the video, it is in this zoom mode that you can clearly see the difference between a 6K video and a 12K video in 360º VR. With this function, even low-resolution displays/screens can benefit from a video that exceeds the resolution of the panel.
Why do 12K today?
With limitations in hardware and networking, and with most TVs only just achieving 4K, comes the question, “why do we even need 12K or zooming?”. The answer lies in the human eye. The human eye sees at approximately 16K in the real world, which is close to 20/20 vision. Anything below this will appear blurry, and blurriness will take away from a fully immersive and interactive experience. By achieving 12K, we are taking the steps necessary to reach the acuity of the human eye.
Though most of today’s VR devices can only support up to 6K resolution, the ability to decode and display a 12K video with zooming allows users to get intimate with intricate details in VR videos. This unlocks a whole new user experience for viewing 360-degree VR video, as well as new possibilities for content creation.
Today, there is a lack of commercial ready cameras that can shoot 12K. However, content creators already have the capability to make customized camera rigs to shoot 12K VR videos. Additionally, stitching together nine 4K videos or four 6K videos would give the same effect as a 12K video. CG content, which is usually created at 12K resolution, can not also be zoomed into without distortion.
Potential use cases for 12K and zooming range from sporting events that allow viewers to focus in on plays and specific athletes, to city tours that allow guests to look closely at storefronts and landmarks; and medical experiences, such as those training for surgery, can now fully express the details and minutia of procedures. Education and travel in general can now be more interactive, and details we once ignored can be brought to light.
How We Streamed and Displayed 12K
There are multiple hurdles to streaming 12K in VR today. One of those hurdles is bandwidth limitations, and a common approach to solving it is improving networking technology and increasing bandwidth, such as 5G. However, experts predict that 5G will not penetrate the mass market until 2026. Additionally, 5G doesn’t address the current limitations of today’s VR hardware. Existing hardware codecs cannot go beyond 4K resolution with traditional streaming solutions, and most VR displays are limited. On today’s VR headset screens or a 2.5K mobile screen, you cannot tell the difference between a 6K, 8K or 12K video.
At Visbit, we invented Visbit View-Optimized Streaming (VVOS) to bring the best streaming and playing experience for 360º VR videos. This technology allows foveated streaming with little to no latency and dramatically reduce the bandwidth needed to stream large 360 VR videos. It is with VVOS that we were able to stream the 12K x 6K video. Since we are only streaming what the viewer is seeing at high resolution we minimize the network toll allowing 12K video in 360º to run on today’s networks with existing hardware. Our zooming technology, addresses the display limitations. By allowing users to zoom, content can be viewed at 12K. In a lower resolution video, zooming, a feature that would cause blurriness in lower resolution videos. This adds an extra of layer of interactivity and clarity that is missing from today’s VR experiences.
Unlocking a Future with 12K
Though this breakthrough is a huge step towards the future of 360º VR streaming, 12K VR is still a few years out from being consumer ready. It is currently both expensive and difficult to shoot a 12K video with a custom rig, and 12K cameras are rare. In addition to the lack of cameras, you need advanced computer hardware to edit and render 12K videos.
However, as we witnessed from the VR content creators at this year’s SXSW, innovation and creation will continue, and it is with the help of technology that limitation are stretched and broken. Here at Visbit, we will continue to bring that future closer and unlock new possibilities.
Visbit: Quality VR Streaming Made Easy
For more information, visit visbit.co
