Human Vision and Digital Color Perception
The camera lens is like our eyes and the camera sensor is its retina. Despite the analogy, humans and machines perceive how they see things quite differently. Humans see color as the amount of wavelength from light that reflects back to the retina’s “cone cells”. Digital cameras, like all other machines, have different sensors. While we understand color as a sort of natural characteristic of a thing or object, machines don’t have this understanding at all. That is why digital cameras just capture images and it is up to software to process and make sense of it to the human eye.
Human Vision
We have a natural image resolution of 576 MP, which is about 18,000 pixels on the vertical so we don’t see any pixelations since that resolution is so high. There are no cameras that have a resolution that is even close to the human eye, at least not at the moment. Canon has created a 250 MP CMOS sensor prototype, which is the closest to the human eye so far (as of this writing February 2019). The eye can also process 1,000 fps as the theoretical upper limit, though it varies per person depending on eye and brain coordination. People with good eye sight have refresh rates between 200 and 400 Hz, while a UHD TV can hit 240 Hz refresh rates. This does not have anything to do with color though, this is just how the eye perceives the image. Color is actually interpreted not by the eyes, but by the brain.
Color is perceived by the eyes and interpreted by the brain.
When it comes to how machines see color, that is an entirely different story. You can program machines and train their software to detect images and objects. However they do not have any perception of color whatsoever, that is a separate process. Upon capturing an image using an exposure setting, the image is stored in memory until it undergoes post processing. That is when the colors are determined by software since it is nothing but binary 1’s and 0’s in storage.
Machine Vision
Let’s first discuss how digital cameras capture color information from images. This requires the digital camera’s sensor. The sensor is made up of an array of tiny light capturing “photosites”. These are like tiny squares on the surface of the sensor that collects photons from light and store them as electrical signals. The signals are then quantified into digital signals that are recorded on storage media. In order to capture color, a filter is actually placed on the sensor which uses 3 primary color: Red, Green, Blue also called the RGB color scheme. These colors are processed according to their “channel”, and in digital imaging each pixel contains 3 channels with represented by the 3 colors (Red channel, Green channel, Blue channel).
Color is perceived by the sensor and interpreted by software.
Color Filters
A “Bayer Array” filter is placed on the sensor that consists of alternating rows of color pairings, namely Red-Green (RG) and Green-Blue (GB). The Bayer Array is common on digital camera sensors, but there are also other types used. Now the Bayer Array has 1 more representation of the color Green compared to Red and Blue. This is because according to science, the human eye is more sensitive to green light so the sensor was created similarly.
According to color theory, the Green channel contains the most detail and least noise among the colors. The Red channel is mainly associated with contrast while the Blue channel is considered the noisiest. Noise in the Blue channel is produced by its physical characteristics. Of the 3 colors, Blue has the shortest wavelength of light.
Light with longer wavelengths is absorbed more quickly than that with shorter wavelengths.
Less noise in an image is due to better light absorption. An image has the least noise when the lighting is great because more photons are perceived by the sensor from the light. Less light means more noise in the image, so it just so happens that it occurs in the Blue channel more often because of its characteristics.
Digital Color Perception
Now we can further explain how we see the colors in the final image from the camera. Most images are shot in unprocessed, uncompressed format. An algorithm must now process the image using the “Bayer Demosaicing” process to reproduce the colors in the final image. Information is extracted from the color array according to the rules set in the algorithm. The final image created consists of pixels (picture elements) which are stored in digital media e.g. SD card, harddrive, etc.
Earlier we mentioned that each pixel contains 3 channels of colors, RGB. In reality there are more than 3, that is why they are referred to as primary colors. You can come up with many color combinations based on blending of different colors which also produce different transitions called gradients. The range of hues in colors one sees is referred to as the gamut, and more colors means more visual appeal. For example when you combine Red and Green it produces the color Yellow. While a machine actually can perceive more colors than the human eye, in reality we cannot see that many colors. According to some research, the human eye can perceive a maximum of 7M colors while machines i.e. computers, can “see” much more. However, that is just a matter of perspective since not everyone sees colors the same way.
The bit depth is what determines the color information of a digital image. The more bits stored in a pixel, the more information is stored and the greater the detail in color. The following is a guide of how many colors can be represented based on the number of bits.
1 Bit — 2 colors (Monochrome)
8 Bit — 256 colors (Low Color)
16 Bit — 65536 colors (High Color)
24 Bit — 16777216 colors (True Color)
The best color reproduction will depend on the display e.g. computer monitor, screen, digital display. OLED displays, like that used on smartphones, tend to have the best color reproduction in a display. IPS backlit LCD panels also have good color reproduction, but not as vibrant as OLED. The image however, must be high resolution in order for the display to reproduce the colors.
Further processing of color in digital images requires applying a color space. Examples of this include DCI-P3, sRGB, adobeRGB and ProPhoto. I won’t discuss this in any further detail, but a color space is basically a description of the range of colors that can be reproduced by a display. There are also cinematic color spaces that provide even more colors at higher resolutions, but the color space discussed here is primarily focused on digital still photography and not video.
Color Is Perspective
Finally, we must discuss color accuracy with respect to digital imaging. Color correction, whether in digital video or photography, is one of the more difficult tasks in post production. That is why you have a DIT or colorist involved in order to reproduce colors more accurately as it should look based on how the director wants to display the image. I think that every photographer has a different way of reproducing their color, even among film photographers. Film itself reproduces colors differently, depending on the characteristics of the film used. This is much like how the sensor works, and having a high resolution camera with a large sensor can capture much more gradients of color with high dynamic range capabilities.
White Balance in photography can also influence how color appears in an image. This is definitely more on human vision, the computer is just there to adjust values to represent the colors in the image. In the end it really boils down to how colorful the image creator wants it to look. Some want to go for a gloomy desaturated look, others go for a warmer appearance while more dramatic moods come out from black and white images. During post, presets and LUTS are usually applied to transform the look and feel of an image based on color and contrast. How we see color in the end is in the eye of the beholder.
