An introduction to 360 cameras
Guillaume Sabran is the CTO and Co-Founder of Pie: the virtual reality video platform for mobile devices and the Samsung Gear VR. Pie is launching a private beta for 360 & VR video creators so if you’re already experimenting with this exciting new medium and want a great community and platform to share your content with sign up to be a beta tester here.
I got the new Ricoh Theta S (a 360 camera I’d highly recommend) for Xmas and a lot of people were curious and interested by what it was, how it worked, and why it was cool. Here are the basics, illustrated.
What does 360 mean here?
All around you, in any direction. i.e. what’s in front, on the side, behind, on top and below. Not just the classic horizontal 360 panorama.
How does it work?
There’s a bunch of camera types, but it’s pretty much the same thing every time: the camera has a number of lenses, from 2 to as many as money can buy, and each of these independently records images that are then stitched together by software to get the final 360 one, for still images and video.
The camera either comes fully mounted or you have to buy a number of independent cameras (generally GoPros), attach them together, and manage the stitching yourself. The last option is clearly the most expensive (6–32 * $500 = $$$$ for the GoPros) and the most painful to get running. Freedom ain’t free. It’s generally used for professional shoots and produces much better output quality … which also means large files, which means it’s a bit of a pain to stream them at the moment.
How do I consume this content?
There are two ways to consume 360 content:
- on traditional screens where you turn either by moving the device (for phones) or using touch/key input.
- in VR headsets. They work pretty much the same (you turn your head to see different angles) but they feel very different because of the immersion.
Which app?
For videos:
- Youtube supports 360 formats on its traditional apps and web player, excepted on iPhone Safari because Apple wants to controls how you view videos and here that just sucks (I’ve just spent days on a workaround!).
- Pie. I’m developing it :)! We focus on the 360 format and develop for the VR experience first since it’s where it’s truly transformative. Apps are going to be out soon as well as a webplayer. Stay tuned by signing up here if you’re a consumer, or get on the private beta list here if you’re a creator looking for a better way to distribute and share your content.
- Apps made by studios such as VRSE, Littlstar etc. There’re definitively worth checking out.
For pictures:
- There are a few places you can upload pictures like Sphere or Oculus Photo, but none yet has the great social flavor we love with pictures.
It’s an exciting area and there are number of reasons 360 photos are a great opportunity. - This project is cool! http://notlion.github.io/streetview-stereographic
What does the image look like?
This image represents what you’d see if you were in the middle of a ball where the world would be projected onto. So deep down it’s spherical. But all image formats that have been developed are optimized for rectangular things and there’s a number of advantages with working with well supported formats. So the image is going to be saved as a traditional rectangular one — jpg/png… for pics and mp4/ogv… for videos.
The spherical image is then transformed to fit a rectangle similarly to how a map is a transformation of the Earth. There are different popular projections:
- equirectangular. Very straightforward.
- cubic. More efficient.
- little planet. More fun.
- pretty much anything you can think of.
Those projections are both a pain to work with and a great canvas for creativity.
The app you use to see it will then understand how the spherical scene was transformed and reverse engineer it to get back to the sphere, or stay with the transformed image that is sometimes beautiful on its own.
Extra :) The less you transform the original image, the better for image quality and, in videos, for compression. That’s why people are moving to cubic projection, especially for videos where it’s important to get successive frames that are quite similar to get good compression. With the equirectangular projections, if the poles change between two frames, the projection makes this small change appear much bigger and it’s bad because two successive frames will differ by a lot (look at the size difference of the pole between equirectangular and cubic)
What about sound?
Most of the cameras have traditional mono/stereo sound but the landscape is quickly changing to 3D sound. It means that the sound is directionally captured so that I can really get the impression that someone is talking behind me, and that he’ll be talking in front of me if I turn. It’s much better for immersion and more importantly it’s key for orientating the viewer in a 360 movie.
3D sound should be the norm very soon, but there’s no great standard out there yet to build with.
You said 3D?
Yes but no, yet. 360, meaning the viewer choose the point of view, makes traditional 3D images (that is just two images, one per eye) much more complicated. Here’s why: basically, traditional 3D comes from two cameras slightly translated shooting the same scene, like two eyes. Now if you turn your head, you turn your eyes. For the two cameras this means they should have turned as well, but it was impossible to know where you’d look at at the time of the filming.
3D + 360 is hard but it’s going to be solved sometime soon because it’s worth the trouble, and it’s gonna change video as we know it!
So people can approximate 3D from a bunch of cameras by estimating what two virtual cameras would see placed in the middle of a big camera ring. Or they can build light field cameras … That the end game, but we’re not there yet, and we’re years away before getting one at a consumer pricetag.
If you have any questions, you can contact me at gui@pie.video or @guisabran :)
Hope you enjoyed it! We’re always keen to hear from 360 creators so please feel free to get in touch. x
