Getting 3D images from 2D Video
3D cameras are really neat and really simple. They have 2 lenses spaced about as far apart as your eyes are that record 2 slightly different images. If you have a viewer that can display one image in front of each eye your brain sees 3D. I got a 3D camera over Christmas and can’t stop taking photos and video of everything from family to nature to random objects. It never gets old.
Playing with a 3D camera got me thinking about how some 2D movie scenes could contain nearly complete 3D information. A great example is the scene in 12 Angry Men where the judge first instructs the jurors. The camera performs a trucking shot, panning sideways across the jurors, who are sitting relatively still. The film was shot with a 2D camera containing only a single lens, but because the camera is moving sideways you can look a fraction of a second ahead in the film to get a second image in a slightly different location.
In other words, you can view normal 2D film in 3D if:
- The camera moves horizontally at a somewhat constant speed
- No object in the scene is moving too fast compared to the speed of the camera
All that’s needed is to stitch together pairs of images that are a few frames apart. You might think there’s some complicated file format for 3D photos and videos, but most viewers will handle standard .jpg or .mp4 with the 2 images concatenated side-by-side.
I used free software to create 3D files: ffmpeg to extract individual frames from video, a few lines of python and montage to create side-by-side images using a fixed number of frames difference, and ffmpeg to stitch the images back into a video. Then I transferred the video to my phone, placed it in a Google cardboard viewer, and voila, I was seeing it in 3D.
I was surprised by the level of detail in the 3rd dimension and by how real it looked. When you watch the 2D film you can still get some sense of depth because objects in the foreground move faster than those in the background, but when each eye gets a different image it’s completely different. For the scene in 12 Angry Men I used a 600ms offset to get the 2nd image (which was actually the left image because the camera was panning left), but I also tried half and double that and didn’t notice a major difference in how it looked.
A tip if you want to view the image above with Google cardboard/daydream: you don’t need any special app — all you really need to do is display the image on your screen, turn off adaptive brightness and the screen timeout, and then stick your phone into the viewer.
