Planespotting

By Tyler Erickson, Developer Advocate, Google Earth Engine

Recently my colleague Chris Herwig found a “plainbow” in a true-color Sentinel-2 (S-2) image composite:

In the image, a single airplane appears as three colored airplanes with multi-colored contrails. I was curious if I could understand the details of why this effect happens, so I started reading through the online docs for the Sentinel-2 Multispectral Instrument (MSI) sensor. Others have described similar rainbow effects for various sensors, but I’ll focus on S-2’s MSI sensor here.

The MSI uses a push-broom sensor design with CMOS detectors (also used in cell phone cameras) for measuring the visible and near-infrared (VNIR) portions of the electromagnetic spectrum. Incoming light is separated into spectral bands using a stripe filter, resulting in a stack of 10 regions (spectral channel sensors) within the detector. The relative position of each spectral channel sensor results in spectral sensitivity that depends on the viewing angle, and in turn results in an inter-band measurement parallax (i.e. difference in the apparent position of an object viewed along two different lines of sight). The S-2 ground processing system knows the approximate altitude of the ground, which it uses when combining the data from the different strips in the sensor back into a coherent image of the ground during the “orthorectification” step.

The following sketch shows how spectral band parallax works, for a simplified 3-band spectral channel sensor. (Given that a picture is worth a 1000 words, this should be 6 times more informative than the preceding paragraph.)

Spectral band parallax. If you are interested in more details, check out the Sentinel-2 MultiSpectral Instrument (MSI) Overview, particularly Figures 2 and 3 and the surrounding text.

If instead the S-2 ground processing system used the elevation of the aircraft, rather than the ground, the aircraft and contrails would look normal and the land underneath would be “rainbowed”.

The airplane that Chris noticed was over land, which introduces background “noise” which partially obscures the effect. But looking farther south along the coast in the same S-2 datastrip I found another plane flying over water, which makes it easier to see the displacement differences between bands.

Using the Earth Engine Code Editor’s User Interface API, I created an app that displays a series of linked maps, starting with a 3-band color composite (B4/B3/B2) followed by a series of individual spectral bands. Visualization parameters for each band were chosen that increase contrast between the airplane and surrounding water. The bands are ordered by the relative positions of spectral channel sensors on the detector, which correspond to the displacement of the aircraft in the image. The graphic below summarizes my findings, but if you want to interactively experiment with the data visualization, you can access the linked map app or export a video showing the band differences.

Sentinel-2 plainbow (aircraft) example over water. The small circles show the shift in apparent position for eight of the MSI sensor’s spectral bands.

The MSI instrument is composed of a series of 12 detectors (in order to image a swath width of 290 km) arranged in a staggered formation:

The multi-spectral instrument (MSI) consists of 12 staggered and overlapping detectors, each with 10 regions of different spectral band sensitivity. (Image based on information obtained from the Sentinel-2 MultiSpectral (MSI) Instrument Overview)

In addition to the intra-detector spectral band parallax effect (discussed earlier), the staggered positioning of the detectors introduces inter-detector parallax differences, as described in the Sentinel-2 MPC Data Quality Report. This inter-detector effect also can be explored in Earth Engine, by displaying a single band image with visualization parameters that enhance the contrast of the water region.

Contrast enhanced image from the B2 blue band. Point (1) shows an example of where a contrail (bright linear feature) apparently breaks due to inter-detector parallax. Point (2) shows that the contrail’s shadow (dark linear feature) can also be seen, but it does not have the same break between detectors (nor spectral band parallax) because the shadow occurs on the surface where the sensor is focused.

In addition to the parallax effect, the actual position of an object may change while the satellite is moving between positions where the MSI sensor is viewing the same location with a different spectral band. This movement also results as a color shift for moving objects on the Earth’s surface (i.e. no parallax effect), such as the ship in the following animation.

Series of band images for a region containing a moving ship (and contrails from the aircraft discussed earlier). Note the red coloring of the leading edge of the ship and the blue coloring of the trailing edge in the RGB color view.

It is also interesting to note that there appear to be four distinct multicolored contrails in the 3-band (RGB) color image. This occurs because, for this particular case, the parallax matches up with the separation between the two contrails, and results in the additive colors of yellow (red + green) and cyan (green + blue).

Another case that would be interesting to explore is the case of slow-moving aerial objects, like hot air balloons or airships (blimps). I made a brief search for examples, but wasn’t able to find any in the S-2 imagery, so I will leave that as an exercise for the reader…

Graphics by Don McCurdy