Why do all the 360 VR videos today look so pixelated?

High-end VR headsets have been out for almost a year, but there are still many problems while experiencing virtual reality. One major complaint we’ve heard from the users is about the poor video quality. Recently Metavision asked such a question on Quora — Why do all the 360 videos look so bad in terms of quality? When can we expect improvement? I think it’s important to help people fully understand the problem. So, I posted my answers on Quora. It turns out many people found it helpful. So, I am posting on here as well, in case others are also interested.

Early this year, I tried watching a live VR streaming on the Nokia OZO, which looked about 1080p resolution. As I was watching, I noticed the streamed content was almost unrecognizable due to pixelation and its low resolution. There are similar problems when watching VOD videos with a Gear VR or Cardboard. Because of this quality issue today, most publishers recommend that users download to watch the VR videos smoothly. This leads to a long wait time and additional efforts to manage phone storage. This is very frustrating for users.

In my research I found that there are at least three major reasons why most existing 360º videos appear to be in such poor quality:

1. VR videos need to be a HIGHER resolution than regular videos in order to appear sharp

A normal human eye with 20/20 acuity is able to distinguish a maximum~60 pixels in one degree of sight. For example, a regular 48in 1080p monitor at a 6-foot distance or an iPhone 6 screen at 15in distance yields 60 pixels per degree.

A common 360º video format of 1080s has ~6 pixels per degree, which is equivalent to human eye vision of 20/200, which is terrible. This is why 1080p VR videos are seen so blurry.

2. Low-quality VR Cameras (including stitching/compression)

Until recently, it was extremely difficult for filmmakers to create content to a high enough quality that could be used with the first generation of VR goggles. Filmmakers did not have many options for recording high-quality VR ready content, as the necessary equipment — cameras and editing software — was not readily available.

4K is actually the lowest quality acceptable for 360º videos. 6K, 8K, or 12K will be viewed as the best quality. Capturing high resolution is a big challenge with VR cameras. Normally it can be a group of cameras on one rig, but this runs into other problems like:

• Adjusting focus on all cameras simultaneously.
• Significant lens aberrations; if it’s a fisheye or if the lens is a bad design or make.
• Stitching is hard to do correctly.
• If compression is too heavy, there will be a huge loss of information.

3. Outdated streaming solutions

360° VR videos require too much data to transfer over the internet in its current form. For example, a 10 minute 4K video can be as big as 1GB in size, which might take half an hour to an hour to download. Streaming on the other hand, could become the best solution. However, traditional streaming is inefficient for VR videos. It would not be able to deliver a good quality VR videos to the majority due to network bandwidth.

• An average decent 4K video has about ~15Mbps for monoscopic eye or ~30Mbps for stereoscopic. I’ve seen 4K videos that are compressed to a bitrate of 2 or 3Mbps, which makes a video full of artifacts, lack of details, and overall poor quality.
• Less than 30% U.S internet users have 15Mpbs network speed on home WiFi, according to Akamai internet report. Less than 10% have 25 Mbps network.

In conclusion, I expect a significant improvement to happen in the coming year.

1. Cameras are getting better in a short span of time. In 2016, we witnessed numerous 4k-8K VR camera being shipped. To name a couple, the 8K GoPro Omni and the Jaunt camera which was available for external users. In parallel, stitching algorithms are getting better as well.
2. Compression for VR videos. Although video compression is mature, it is reasonable to expect ~25% improvement if optimization/customization is done properly for VR.
3. Next-generation streaming for VR video. The basic idea is since users only stream, for example, 90 degrees then streaming should only occur where the users are looking. This is called View-Optimized Streaming*. A good explanation can be seen here, or here. View-optimized streaming technology is able to dramatically improve quality by 400% or more.
4. Internet Bandwidth. The Internet connection speed is growing steadily by 50% per year according to Nielsen’s law.
5. Screen resolution is increasing. Samsung is rumored to be working on a 4K screen for phones, which would be able to render 360º video at 8K to their users.

Everyday there is continuous progress and growth within the VR industry. My feeling is that some time in 2017, the majority of early adopters will be able to not only watch better quality 4k VR videos, but might even be able to watch 6k or 8k quality VR videos. I project that around 2020, people will start to watch even 12k or 16k, which will be sharp as our physical reality.

My team and I are working hard on the View Optimized Streaming solution, which now can stream from the cloud and play up to 8K resolution 360 VR videos over the current standard WIFI and LTE . There are many other scientists and developers working hard to improve the 360 degree camera, the post-production workflow, as well as the high resolution displaying solution for VR. Hope with our efforts, the day to enjoy truly high quality VR can come soon.