How An Ordinary Camera Phone Can Photograph Objects Hidden Behind Other Things

Physicists have found a clever way to process images from ordinary smartphones that reveals objects hidden behind materials that scatter light

The world of imaging is currently undergoing a quiet revolution. In the last few years, physicists have worked out how to take high resolution images of ordinary scenes using just a single pixel. Yep, that’s just one pixel. Indeed, the same single-pixel technique can produce 3D images and even 3D movies.

If that sounds extraordinary, it’s just the start. This technique requires no lens. Instead, the pixel sits behind a flat sheet of frosted glass that randomises the light from the scene. In a sense, the frosted glass takes the place of a lens.

What is needed, though, is lots of images, which must be processed using a powerful computer to reproduce the original scene.

Now physicists have gone even further. Instead of a single pixel taking lots of images, they show how to do the same thing with lots of pixels taking a single image from behind a piece of frosted glass.

That’s just how ordinary digital camera work. So what these guys have done is show how an image taken with an ordinary smartphone camera can reveal objects that are hidden behind a material that scatters light like frosted glass.

To understand how this works, first some background details about single-pixel imaging. It’s easy to imagine that a single-pixel image taken from behind a piece of frosted glass captures an entirely random pattern of light.

But in fact, a sequence of images taken this way are correlated because they all record the same scene, in a slightly different way.

So the trick to reconstructing the image is to crunch the data from lots of images, looking for the correlation between them. When you find this correlation, it’s a straightforward task to extract an image from it.

And that’s it. One pixel, one piece of frosted glass and one powerful computer. Plus lots of images. Indeed, the more images that are taken, the higher the resolution of the final image.

Now Ori Katz and a few pals at the Institut Langevin in Paris have worked out how to do the same thing with a single picture taken by an ordinary digital camera.

The trick here is to treat the data from each pixel as a separate image. The task then is to look for the correlation between each of these images, just as in the single-pixel imaging techniques.

Of course, in a digital camera, each pixel records the image from a slightly different angle. But even so, they all record the same scene and so produce images that are correlated.

And since the light doesn’t need to be focused, no lens is needed. “The scattering medium effectively serves as the lens,” they say.

That leads to a significant advantage over the earlier technique. Because of the large number of pixels in modern cameras, the imaging process becomes parallel. In fact, Katz and co say the technique parallelises the data acquisition process over a million-fold.

So the new technique consists of taking a single picture of a scene hidden behind some kind of light diffusing barrier and then crunching the data to reveal the original scene.

Katz and co have tested their technique using the camera on a Nokia Lumina 1020 smartphone which has a 41 megapixel sensor. “In these experiments we placed various objects 20cm from the diffuser and simply took an image of the scattered light with the cameraphone placed on the other side of the diffuser,” they say.

There are one or two challenges of course. The lenses on smartphones—not the best in the world to start with—are unnecessary. To reduce their effect, Katz and co place a pinhole aperture in front of the camera lens.

Smartphones also process the data from the pixels, producing a compressed jpeg image. But the team’s technique works even after this kind of processing.

Finally, the correlation between single-pixel images depends on the colour of the light used. So to keep the data processing manageable, the technique requires light of a specific colour. That’s relatively easy to achieve using filters or special light sources.

The results are impressive. Katz and co use their Nokia to produce images of several objects hidden behind light scattering layers, such as frosted glass, onion skin and even chicken breast tissue.

And they even produce images using reflected light (as opposed to transmitting light). To prove this, these guys recorded the light from an object that was scattered off a wall covered in white paint.

Sure enough, the resulting image revealed the object, even though it was essentially around a corner from the camera.

The applications are many. It’s no coincidence that these guys imaged an object behind breast tissue (albeit from a chicken). Medical imaging must be a key area of application.

It should also be useful in situations where lenses are hard to use. The most expensive and problematic part of the Hubble Space Telescope, for example, was its lens. It’s not beyond the realms of possibility that future orbiting observatories—both those that point into space and those that point towards Earth—might have the equivalent of a flat sheet of frosted glass instead.

That should make them smaller, lighter and considerably cheaper. All qualities that are highly sought after by researchers, commercial satellite operators and, of course, governments.

Clearly, a revolution in the offing.

Ref: arxiv.org/abs/1403.3316 : Non-Invasive Real-Time Imaging Through Scattering Layers And Around Corners Via Speckle Correlations

Follow the Physics arXiv Blog by hitting the Follow button below, on Twitter at @arxivblog and now also on Facebook