Will VR feel like reality soon?

Current VR (virtual reality) headsets are undeniably impressive pieces of immersive hardware. Now that several consumer-grade headsets are on the market, brands are thinking about potential use cases for VR, whether using 360-degree videos or real-time computer graphics. If you’ve had the opportunity to try either, you will have noticed that VR experiences rely on the immersive aspect but that the visual quality still falls short of what would be ideal for visually demanding marketing purposes.

Despite the amazing technological progress that has been accomplished, we are not close to the ability to simulate activities that would require good visual acuity or provide an experience matching our natural vision.

To understand why, let’s see how the display of an VR headset works.

A small screen is placed in front of a Fresnel lens that bends the light so that we don’t notice that the screen is 5 cm (2'’) away from our eyes. Each eye actually sees half of the screen. The current VR headset resurgence has been made possible thanks to the ubiquity of high-density screens used in mobile phones. The current generation of headsets uses screens that are close to WQHD (1440p) with a display resolution of 2560x1440 pixels, which gives 1280x1440 pixels per eye.

The horizontal FOV of current headsets is around 110 degrees. This number has not been chosen by chance: it is our natural binocular vision field of view. Current headsets forgo completely the peripheral vision (220 degrees).

How does this compare to human vision?

The normal visual acuity of an adult is close to 1 arc minute for 20/20 vision (many people have better vision than that). Given a 110-degree FOV, a VR headset would need to be 6600 pixels wide for each eye. We are currently at around 1300. In effect, when you are wearing a current-gen HMD headset, you have myopia.

Many virtual activities for which a VR headset would be amazing require good visual acuity: driving a sports car, flying a plane, looking into a rifle’s scope, viewing high-resolution 360-degree videos.

In the United States, the minimal visual acuity required to legally drive a car is 20/40, which would be a good target to call a headset “simulation-ready.” Such a headset would have to display about 3300 pixels per eye.

When are we going to get those 6600 pixel-wide headset then?

The current gen uses 5’5'’ WQHD screens, but better screens are already on the market. In November 2015, Sony released the Experia Z5, which uses a very high 4K resolution 5’5’’ screen (806 dpi!). A headset using such a screen would provide 1920 pixels per eye. Not bad but still quite far from the minimum 3300 we want.

8K screens are already being developed (though they are currently much bigger than 5’5’’) and provide a resolution of 7680 x 4320.

When an 8K screen can reasonably fit in a headset, we will get a resolution of 3840 pixels per eye, which corresponds to a visual acuity of 20/34. We would still be slightly visually impaired, but it would be good enough for most VR activities.

Unfortunately, we are not going to see such headsets on the market for a while, as 8K screens small enough to fit in an HMD are not yet on the market. However, it is interesting to note that Samsung is working on a 11K mobile display set to come out around 2020 (project EnDK) that would allow near human-level visual acuity in a VR headset.

Can’t we have it before 2020?

If you don’t mind a clunky headset, a solution that could be engineered today would be to create a large headset using two 4K 5’’5’ screens (one for each eye) in a similar fashion to the StarVR headset. In addition to the extra weight and size, it also comes at the cost of vertical FOV, which would be limited to about 60 degrees (human vertical FOV is 135 degrees).

The thing is that the display resolution discussion is moot, as even if a screen manufacturer released a suitable 8K screen next year or someone makes a double 4K screen headset, it cannot make it as a successful consumer product if the computer system required to drive it is too expensive.

VR requires stereoscopy and a fast frame rate: with the current technology, performance requirements for VR are around 4x times as much as for a regular game rendering at 1080p/60fps. Creators of VR experiences already have to take this into considerations and limit the usage of state-of-the-art computer graphics in their work today.

Buying a computer powerful enough to output 3D graphics in stereoscopy at 4K+ resolution would be prohibitively expensive for regular consumers. In fact, it might not even be possible with the consumer graphics cards on the market today.

Is it all bad news?

No! Foveated rendering is being developed and will allow creator to avoid having to render the whole scene at max resolution and should lower the required processing power by two to four times. This is the breakthrough that will probably make 4K headsets viable consumer products. But the truth is that display technology will probably keep moving faster than computer graphics.

This should not prevent you from buying a headset today if you want it: the immersive experience can be great today, but if you were waiting for VR to be ready for near-human-level visual experience, don’t hold your breath… You will probably have to wait a while before it’s possible at a reasonable price point.