The Basics Of Dual Pixel Imaging

Digital Sensor Photosites

Have you seen the latest smartphone cameras? They are creating amazing images that rival DSLR cameras. They look like they were shot by a professional, but it was taken by an ordinary person. The technology behind today’s smartphone cameras combines traditional optics with software algorithms. This is a new form of imaging called computational photography. One of the most well known examples of it is called dual pixel technology which will be further explained.

Example of a photo taken using dual-pixel sensors from a Pixel 3. Take a look at the bokeh effect created in this photo. (Photo Source Google Pixel Gallery)

Traditional cameras have always been optic based. You have a lens to focus and create the composition of the image and then you capture it on film to create an exposure. The introduction of digital cameras replaced the film with an electronic sensor, but still uses optical imaging to get the final result. With computational photography, it adds more to image enhancement using in-camera capabilities. It is as simple as point-shoot-click of a digital camera, but there is more involved with the image processing to create the final image.

A brief illustration of dual-pixel sensors

The dual pixel sensor addresses a problem smartphone camera designers faced with sensor size. A smaller sensor was necessary on smartphones in order to fit in the smaller sized body. A DSLR or mirrorless camera have bigger body sizes to fit large sensors. Larger sensors can capture more light and thus produce better quality images. How can this be done on a smartphone, where you will not have enough space to fit a high quality image sensor? The answer from some smartphone vendors is to use dual pixel sensors. These digital sensors are not just capturing images from light, but they are capturing values which are then processed and refined by the camera software.

This diagram shows how the pixel is created from the two photodiodes from the photosite.

Sensors consist of photosites which represent the pixel or picture element in an image. These are light capturing devices called photodiodes, and they cover the surface of the sensor. The more light the photosites capture, the better quality in resolving the image. In a smartphone’s dual pixel sensor, the photosites are much smaller, and so by default do not capture as much light as a full sized camera sensor e.g. CMOS 35mm sensor. Instead what dual pixel sensors do is divide the photosite into two separate photodiodes. When the lens starts focusing on the image, each photodiode captures light at different angles and receives different inputs of the image as well. This acts similar to phase detection autofocus on a DSLR camera. The two photodiodes in the photosite now combine their values to create a single pixel output. The photosite’s values are then processed by complex algorithms to create the final image.

By combining values from two photodiodes in the sensor’s pixel photosite, you get the result of more detail when it is combined into a single pixel. This was shot from a Pixel 3 (Photo Credit Android Authority)

Combining the dual pixel sensor with the lens, you get a camera that is capable of shooting stunning images in different modes and lighting conditions. With AI techniques used in machine learning, the photo effects are created using training data from millions of photos to create the best image. That is how an untrained user who has no knowledge or experience in photography can take a great photo. There is no need to worry about the aperture, shutter speed, white balance or even the ISO settings. The composition and creativity is in the user, but the final image created was done by the smartphone camera system.

The portrait modes in some smartphones take really great selfies because of dual pixel sensors. Google has really made good use of this in the Pixel 2 smartphone line. Despite having only 1 main camera, it was able to produce better results than its dual-camera competitors like the Samsung Galaxy and Apple iPhone. The dual pixel sensors measure the difference from one side of the pixel and the other rather than using two lenses. This proves that you don’t really need more than 1 camera on a smartphone to take great images. Google does this with AI techniques in software and dual pixel sensors. One feature that makes Google’s smartphone cameras take great photos is the focus, and that has to do with the dual pixel autofocusing found on the Pixel 3 line.

Smartphone camera makers actually borrowed dual pixel technology from the DSLR. Canon is credited as the first camera company to introduce this in their Canon EOS 70D called the Dual Pixel Autofocus feature. The EOS 70D still used phase detect autofocus, but it won’t rely on the camera systems mirrors. Instead the sensor will handle the autofocusing of light without the need for other modules. This is exactly what a mirrorless system like a smartphone camera needs. Without having to rely on mirrors, which require mechanical motion to adjust and thus takes longer to focus, a dual pixel sensor is more precise and can focus on the image faster.

How Canon implements their dual pixel sensor illustrated (Source Canon)

It is now clear, dual pixel sensors main benefits are fast autofocus and sharper imaging. The camera can take accurate focus on an image regardless of its position. Even DSLR and mirrorless cameras use dual pixel sensors, but to incorporate this in a smartphone camera is very practical due to how it enhances images despite its smaller size. Smartphone vendors will surely come up with variations of this technique combined with their AI software to allow users to take their best photos and stunning images.

--

--

--

Multimedia, Imaging, Audio and Broadcast Technology

Recommended from Medium

How to Turn off Automatic Device Switching on Airpods

Coming of Age: Why You Need to Build for Smartwatches

Everything You Need To Know About Mobile Payments Technology

The Samsung Galaxy S10e Camera In Action

Spatial Media As Reflected by Technological Advances

Why did I ever sell my Porsche 911e?

ASML sets GUINNESS WORLD RECORDS™ title for the Smallest advertisement

Tesla Diner + Drive-In Theatre — WHAT!?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Vincent Tabora

Vincent Tabora

Editor HD-PRO, DevOps Trusterras (Cybersecurity, Blockchain, Software Development, Engineering, Photography, Technology)

More from Medium

See how the first ever pic of a black hole was obtained.

A star safari in Greater Kruger

Will the iPhone 14 Talk to Satellites?

Image Manipulation: The What, How, and Why