Comparing 360 Video Formats
Two weeks ago, Google announced a new 360 video format that improves image quality in areas where users spend the most time looking, and makes the overall image quality more uniform. We decided to see how the assumptions behind this new format compared to those of the two other popular formats, using real viewer data from the over 10 million 360 video views we’ve analyzed at Vrtigo.
In this post, we’ll discuss our approach to measuring the performance of 360 video storage formats. In the next post, we’ll use data from actual viewer sessions to compare the performance of each format.
How 360 videos are stored
To measure the performance of different storage formats, we first need to understand how 360 videos are stored. Most 360 videos exist as “flat” 2D videos that are texture-mapped onto a 3D object in VR-space, centered around the viewer’s head. So the viewer is looking at a virtual object that has a video projected onto it.
As described in Google’s post, the ideal 360 video format should space the pixels out in the 2D video file such that when they’re projected onto a sphere or a cube and back onto the headset’s display, an efficient number of pixels are being used. So if a feature in the video occupies 100 pixels in a user’s VR headset, it should only require 100 pixels of storage in the video file. Any more is a waste of space and bandwidth, and any less reduces image quality.
The two most popular formats project a rectangular image onto a sphere or cube and then render it in the headset, causing distortions that make the pixel spacing uneven. The equirectangular projection places too many pixels above and below the viewer, and not enough in front of them where they’re most likely to be looking. Cubemaps are an improvement, but they still have too many pixels in the corners and not enough at the faces.
The Equi-Angular Cubemap (EAC) introduced by Google also projects onto a cube, but places more pixels at the centers of the faces so that the end user sees a more uniform pixel coverage. They also introduced a useful way to analyze the uniformity of this spacing by plotting saturation maps of the ratio of video pixel density to display pixel density for each direction the viewer is looking. We will call this the Pixel Density Ratio (PDR). So for a given set of yaw and pitch angles, (𝜃, 𝜙), we will denote the ratio of video pixels to display pixels in that direction as PDR(𝜃, 𝜙).
Quantifying viewer session quality
To create a good viewing experience, the video should display the optimal number of pixels in every direction the viewer looks. Another way of saying this is that the ratio PDR(𝜃i, 𝜙i) should be as close to 1 as possible for every sampled view direction (𝜃i, 𝜙i).
So for a given view session, the distribution of the PDR values indicates whether the viewer spent most of their time viewing areas with high or low pixel density ratios. We can measure this distribution using simple statistics like mean and variance:
It’s worth noting that a high mean value is not necessarily optimal, since values greater than 1 indicate that too many pixels were stored in that part of the image. So both the mean and standard deviation should be considered as a pair when evaluating viewer experience.
To compute these statistics, we formulated the PDRs for equirectangular, cubemap, and EAC projections based on the projection geometry and some assumptions about video and display resolutions, coordinate systems, etc. These assumptions will vary in practice, so we attempted to choose formulas that best represented differences in uniformity between the formats.
In our next post, we’ll use this framework to analyze actual viewer sessions to see how much difference these improvements in storage formats make for real users.
Note: Some characters used in this post don’t display correctly in mobile browsers. View in a desktop browser to see all the formulas.
Vrtigo is the next generation VR analytics platform for content creators, editors, producers, and marketers.
