Locating the usable data in Planet imagery
On average approximately 70% of the Earth’s surface is obscured by some level of cloud or haze, meaning that it can be a challenge to get a clear view of an area when using optical satellite imagery. Fortunately, Planet’s Dove constellation is able to image even the world’s cloudiest areas due to its rapid revisit rate: if six days a week are overcast, the satellites that pass overhead every day will have the best chance of capturing a cloud-free image.
Cloud filters and cloud masks
As each Planet image passes through our automated imagery pipeline, we estimate the percentage of pixels in the image that are obscured by cloud, generating a final score that is added to our metadata.
Planet Explorer and Planet’s Data API allow users to filter imagery search results on how cloudy the image is. In the sample image below, the metadata indicates that it is approximately 74% covered in cloud: "cloud_cover": 0.743.
If the user had selected a cloud-cover threshold of 74% or less, this image would not have appeared as a result when searching through Planet’s imagery catalog.
In addition to providing cloud statistics on a per-image basis, Planet also provides a way to find out whether each pixel is cloudy or not. To do this, Planet provides a downloadable asset named udm—the Usable Data Mask. A udm is automatically included alongside the imagery file when you order analytic data in Planet Explorer.
In the image to the left, the automatically-detected cloudy pixels are highlighted in yellow.
If you do want to test a udm yourself, note that file has a unique structure that is not always the easiest to dissect (see the latest version of the Planet imagery specification on our Documentation site for details).
To the human eye, udms do a decent job at detecting what are clearly clouds but they also pick up degrees of haze that exist elsewhere in the image. And while the cloudy pixels can probably be safely ignored, some of the “masked” pixels have usable data that is only slightly obscured by haze.
Enter the new cloud mask
With the introduction of the new udm2 asset, Planet users gain a number of new possibilities around both filtering images and classifying pixels that we believe are a significant improvement.
The new UDM has a very different structure from its predecessor; instead of packing all the information into a single-band GeoTIFF, udm2 is composed of 8 different bands.
Bands one through six indicate where a pixel has been classified in one of the following ways:
- clear
- snow
- shadow
- light haze
- heavy haze
- cloud
For example, when a pixel has a value of 1 in the sixth band, it is a cloudy pixel. If the third band contains the 1, then the pixel is considered to be affected by a cloud shadow.
One way to make use of the udm2 asset is to mask out pixels in the image that you do not want to use as part of analysis. For example, in the image above, we might want to remove the pixels that were classified as being cloud, heavy haze or shadow, but retain the pixels with light haze since we believe the data will still be usable for our needs.
import rasteriowith rasterio.open(img_file) as src:
profile = src.profile
img_data = src.read(masked=True)
with rasterio.open(udm_file) as src:
shadow_mask = src.read(3).astype(bool) # band 3, shadow
hhaze_mask = src.read(5).astype(bool) # band 5, heavy haze
cloud_mask = src.read(6).astype(bool) # band 6, cloudmask = shadow_mask + hhaze_mask + cloud_maskimg_data.mask = mask
img_data = img_data.filled(fill_value=0)
Another nice little trick is to “collapse” the five bands that show the class into a single band, something which makes it much easier to visualize the UDM as a single layer in a GIS.
import rasteriowith rasterio.open(udm_file) as src:
data = src.read()
profile = src.profilemask = ((data[1] == 1) * 2 + # snow/ice = 2
(data[2] == 1) * 3 + # shadow = 3
(data[3] == 1) * 4 + # light haze = 4
(data[4] == 1) * 5 + # heavy haze = 5
(data[5] == 1) * 6) # cloud = 6profile.update(count=1)
profile.update(nodata=0)
profile.update(compress="deflate")
profile.update(interleave="pixel")
with rasterio.open(new_udm_file, "w", **profile) as dst:
dst.write(mask.astype(rasterio.uint8), 1)
The following code will produce a one-band GeoTIFF file that contains a pixel value of 0 unless something was detected in that location. When something was found, a different numerical value will indicate the specific class.
Loading this new GeoTIFF into a GIS and assigning different colours to each classification type, we can quickly see how well Planet’s new cloud mask worked.
Better filtering too
In addition to the udm2 asset, users now have access to several new metadata fields to allow for more precise filtering of imagery when searching Planet’s catalog.
Most of the new additions to PlanetScope imagery metadata match the bands in udm2: clear_percent , cloud_percent, snow_ice_percent, heavy_haze_percent, light_haze_percent and cloud_percent.
Using Planet’s Python client, you can find images that are at least 90% clear (gte means “greater than or equal”) :
planet data search --item-type PSScene4Band --range clear_percent gte 90We hope that this short introduction will help users locate imagery that will best suit their needs. You can read more about using these new features in our developer guide and API documentation.
See also the Launch Programs page for information on how Planet engineers can help you get the most out of Planet’s imagery and platform through training, best practices, guidance and technical assistance for your implementation team.
