Immersive Video: Which camera is right?
Immersive Video (fka "3D VR180") is an exciting new medium teleporting the viewer to the heart of the action. The Apple Vision Pro's stunning 23 million pixel display, paired with state of the art spatial audio, unlocks the viewing quality required for it, delivering 180 degrees of crisp visuals and unparalleled depth. I argued recently that, more so than spatial experiences, it is the Vision Pro's killer app.
This week's yearly Apple developer conference, WWDC 2024, brought its lot of announcements surrounding the Vision Pro, its launch in 8 more countries, updates to visionOS, as well as new immersive video content. With Apple's own productions being shot with the secretive and proprietary camera pictured above, the most exciting announcements were in the hardware department, with new camera equipment geared towards immersive video. This makes now (June 2024) the perfect time to go over all the hardware options for immersive video filmmakers.
Top of the line & Professional range
While the field of immersive video is still experimental, consensus over the minimum requirements to replicate the quality of Apple's immersive videos is starting to emerge among early adopters: at least 8K (4320 pixels for each eye), 90 frames per second to match the display refresh rate, 180 degrees of field of view. At the time of the Vision Pro's release, Canon declared that no available camera is good enough. However, things are changing at a fast pace.
Canon RF 5.2mm F2.8 L Dual Fisheye Lens
Before we dive into cameras properly speaking, let's go over Canon’s dual fisheye lens, as it will be used as a reference point throughout.
While arguably not a high-end product, it has established itself as the go-to optics for Immersive Video. Launched two years ago for the mirrorless, full-frame EOS R5 and R6 hybrid camera families, it was designed with the lower resolution of the Meta Quest headsets in mind.
The optics are split between two fisheye lenses 60mm apart, mimicking the average human interpuppilary distance. This projects two circular, 190-degrees images side-by-side onto the camera's sensor. The field of view is so extreme that the lenses are in each other's frame.
The main advantage of such a dual lens is that it streamlines calibration in post-production: no need to time-sync two cameras, no need to balance colors, no need to keep focus in sync, no need to measure camera positions. A lot of that work is done "for free" thanks to the left and right eyes landing on the same sensor.
This lens lacks auto-focus, however for immersive video you'll want to use the smallest viable aperture (larger f number), which lets less light in but maximizes the depth of field.
The lens retails at $2000.
(Just Announced!) Blackmagic URSA Cine Immersive
I mean, just look at this beauty! This new camera launching later in 2024 boasts 8160 x 7200 per eye at 90 FPS, with 16 stops of dynamic range, hinting at great low light performance.
MSRP was not part of the announcement, but a ballpark price can be inferred from the recently-released, $15K Blackmagic URSA Cine, which unlike the Immersive comes with a wide range of lens adapters (but not the RF lens that would support Canon's dual fisheye lens).
Blackmagic Design also announced an update to its popular editing tool DaVinci Resolve Studio promising an all-batteries-included post-production workflow designed for the Apple Vision Pro. As post-production software is called out as the main pain point by immersive videomakers, this is promising.
RED V-Raptor [X]
At time of writing the only mainstream solution for Immersive Video capture, the RED V-Raptor captures in 8K at up to 150 FPS, with 17 stops of dynamic range.
While it does not come with its own stereoscopic optics, its RF lens mount adapter makes it compatible with Canon's dual fisheye lens, as pictured above. While the RED V-Raptor retails at an eye-watering $30K for the box alone, the adapter is an extra $1500.
Post production software does not currently support immersive video or the Vision Pro as a target out of the box, so be ready to use a combination of RED's own software as well as specialized VR calibration software like Mistika VR, before even jumping to editing. Technically capable of live streaming, the RED V-Raptor requires a ridiculously complex and expensive setup.
While this setup is favored by multiple VR180 filmmakers, no footage is available on the Vision Pro at time of writing.
Canon EOS R5 C
The go-to Canon camera to pair with the dual fisheye lens is certainly the hybrid EOS R5 C, released two years ago, where C stands for Cinema. The two products are bundled in Canon's VR Content Creator Kit. This camera can record at 60 frames per second at 8K, with 12 stops of dynamic range, coming close to the minimum requirements for immersive video. What makes the camera hybrid, other than its recording capability, is the "dual firmware" switching between EOS photo and video modes, with more options and controls than the EOS R5. Also, an onboard fan alleviates overheating issues found on the R5.
Interestingly, the EOS R5 C's onboard battery is not capable of recording at 8K 60 FPS; the camera needs to be powered by either a V-mount battery, a USB-C power bank or a USB-C laptop charger.
Canon offers subscription-based software for VR calibration in the form of either a stand-alone utility or a plugin for Adobe Premiere.
I have yet to experience the "vanilla" EOS R5 C image quality on the Apple Vision Pro. Videos produced by Canon and Meta look great on the Quest 3, which makes me cautiously optimistic.
According to his reddit thread, the footage in James Hustler’s Explore POV app was captured with the EOS R5 C. However he ran a custom upsampling pipeline to 16K in order to achieve a visual quality close to Apple’s own productions.
The EOS R5 C retails at $3600 for the box alone, and $5100 bundled with the dual fisheye lens.
Prosumer range
Immersive Video is not a brand new concept: while it is receiving renewed attention with the Apple Vision Pro, its earlier avatar (called 3D VR180) is on the tail end of a wave started 2–3 years ago with the Meta Quest and Pico VR launching and bringing affordable 2k pixels per eye graphics to the mass market. The jury is still out whether those cameras can produce immersive video quality good enough not to be distracting to the viewer, however, while not cheap, they are much more affordable.
(Just Announced!) Canon RF-S 3.9mm F3.5 STM Dual Fisheye Lens & EOS R7
At first glance, the new RF-S 3.9mm F3.5 STM dual fisheye lens from Canon, designed for the EOS R7 camera doesn't look that different from its elder sibling the RF 5.2mm F2.8 L dual fisheye (detailed above), and you would be forgiven for thinking that the main difference is support for full-frame vs. cropped-frame APS-C cameras.
Unlike the full frame version, this new lens features an STM autofocus motor and offers a narrower 144 degree field of view, meaning more pixels on the sensor for the part that matters most right in front, and a simpler setup since it won't require an Alexander Calder-style tripod to keep the feet off the frame.
There are similarities, such as an interpupillary distance of 60mm, and calibration support through EOS VR software (standalone utility and Adobe Premiere Pro plugin).
The Canon EOS R7 camera however is a more mid-range, versatile, hobbyist box than the EOS R5 C. With 10 stops of dynamic range, it captures up to 4K at 60 FPS, meaning only 2K per eye, which won't amount to the same crispness, even accounting for the tighter field of view, as video captured on the EOS R5 C.
The EOS R7 launched two years ago and currently retails for $1400. The new dual fisheye lens starts shipping at the end of June and retails for $1100. Bought together, they're about half the price of their EOS R5 C + dual fisheye lens equivalent, a bet by Canon that a market exists for lower resolution, lower cost immersive video.
Kandao VR Cam
Unlike most other cameras detailed here, the Kandao VR Cam is designed for live-streaming first. Boasting an 8K resolution and 30 frames per second, it hooks up via Ethernet to a Windows machine running Kandao software to calibrate, compress and stream (or record) the video on the fly. Interestingly, it uses AI to figure out color correction and parallax alignment. As you can guess, a powerful GPU is needed to run all this. At 65mm, the interpupillary distance is slightly wider than Canon’s.
Hugh Hou's video review shows that Kandao VR Cam's image quality comes closest to Canon's EOS R5 C, in spite of an M43 sensor that's even smaller than APS-C. This is due in part to its pair of removable Laowa 4mm f/2.8 fisheye lenses.
In my own tests based on their sample videos, I don't think the image quality is high enough for the Apple Vision Pro. And I don't think the focus on live-streaming currently justifies the retail price of $4000 over alternatives.
TECHE 3D180VR Camera
Released last year, the TECHE 3D180VR Camera is also designed for live streaming in 8K at 30 frames per second. Unlike its competitors, it is capable of broadcasting to the cloud by itself, without requiring a powerful computer to process and stream the video.
Its onboard computer runs AI to auto-calibrate the video, but according to Hugh Hou's review from last year, it performs poorly enough to warrant manual stitching, requiring a powerful computer after all. And because its image quality seems to compare negatively to Kandao, I don't see this being good enough for the Apple Vision Pro. The narrow use case I think could justify its $4225 price point would be streaming on the go.
CALF 3D VR180 Camera
CALF launch its 3D VR180 camera last year, boasting a 6K resolution captured at 50 FPS and a field of view of 185 degrees. At 65mm, the interpupillary distance is wider than Canon and the same as other VR boxes.
CALF's main value proposition is a balance between ease of use and video quality: you press the record button and CALF's software facilitates editing, on-demand distribution on various VR platforms, and even live streaming. Retailing at $1800, its price is competitive with the new Canon EOS R7-based solution. CALF's VR player has not yet launched on visionOS, but Hugh Hou showed the CALF image quality compares negatively with both Kandao and TECHE, because of its lower resolution and more aggressive sharpening and compression.
Marketed as a professional camera, the CALF 3D VR180 is actually the entry-level option.
Spatial Video range
A common refrain among photo and video professionals I've talked to is the confusion between Immersive Video and Spatial Video.
Both Immersive and Spatial video are meant to be stereoscopic 3D video representing depth at a more or less 1:1 scale from the viewer's perspective. Immersive "surrounds" the viewer at 180 degrees or close to 180 degrees, while Spatial shows the content in a smaller, "punched in" window. Think of Spatial as a 3D TV, and of Immersive as the Sphere in Las Vegas.
The big advantage of Spatial over Immersive is that it can be captured and played natively in the Apple Vision Pro using the Camera and Photo apps. But because of its smaller format, Spatial is less immersive than, well, Immersive. Its depth effect is nevertheless still compelling vs. flat photos, which is why Apple and now Canon bet its adoption will catch on.
(Just Announced!) Canon RF-S 7.8mm F4 STM Dual Lens
Coming later this fall and designed for the aforementioned Canon EOS R7 cropped-frame APS-C camera, this lens will launch with an updated EOS VR utility that can convert raw footage to Apple Vision Pro's MV-HEVC Spatial Video format. The lens will sport a motorized autofocus.
Calculating the horizontal field of view from (half of the) APS-C sensor size and focal length suggests a horizontal field of view of 70.87 degrees, similar to the Apple Vision Pro's spatial videos.
No MSRP has been announced, but if the RF-S 3.9mm F3.5 STM dual fisheye lens is any guidance, one can expect a $1000 ballpark.
Apple iPhone 15 Pro and iPhone 15 Pro Max
Since the release of the Apple Vision Pro this February, both variants of last year's iPhone 15 Pro can record landscape mode spatial videos in the camera app, in 1080p per eye, 30 FPS, 63 degrees of field of view. Lacking the appropriate camera setup, the non-Pro iPhone 15 and iPhone 15 Plus do not offer the feature.
At time of writing, the iPhone 15 Pro and iPhone 15 Pro Max retail from $1000 and $1200, respectively, and are expected to be discounted with the launch of the next generation in September/October.
Apple Vision Pro
Unsurprisingly, the Apple Vision Pro itself can also record square spatial videos, in 2K per eye, 30 FPS, 71 degrees of field of view. The camera app is even unable to capture in 2D!
No need to introduce it further, the Apple Vision Pro retails from $3500.
Conclusion
While the actual performance, price, and availability date of the newly announced Blackmagic URSA Cine Immersive camera remain to be seen, chances are it will be a cheaper and better alternative to the current best in class RED V-Raptor with Canon dual fisheye lens.
In the meantime, the Canon EOS R5 C with dual fisheye lens is likely to remain the best mid-range option.
At the entry level, I am looking forward to the upcoming release of Canon's APS-C dual fisheye lens to see if, paired with the EOS R7, 4K looks better than 8K at 30 FPS from all-in-one VR boxes like the Kandao VR Cam and the CALF 3D VR180. I'm not holding my breath, though, as I don't see how it could overcome its low pixel count and deliver the crispness that Apple Vision Pro users have come to expect.
Finally, it's interesting to see Spatial Video get its first big boy lens later this year with the Canon APS-C spatial video dual lens. Will the relative user-friendly nature of the format drive adoption? Or will it pale too much compared to Immersive?
Shoutout to Hugh Hou's YouTube channel and his very thorough reviews.
Corrections:
- June 17: Explore POV footage was shot with the EOS R5 C and then upscaled, not shot with the RED V-Raptor as previously stated.
