Remote Sensing 🌎 + ML 🤖 = 🤤

This is perhaps my most random blog post subject yet but I personally find it really cool. ML + remote sensing is a great combination because it 1. Tackles really important issues and 2. The abundance of remote sensing data makes it a well suited ML task. In this post I will describe some cool ML + remote sensing applications with link to related paper and useful tools if you want to create your own project.

Precursor Question: What is remote sensing?
Answer: Remote sensing is the collection of information about an object or phenomenon, such as land surface temperature, without direct physical contact, typically using sensors on aircraft or satellites to gather data from a distance

Several Applications

Planted Forest Classification

Spatial distribution of dominant tree species to the genus level across the planted forest range in East Asia from Fig. 8 of relevant paper below

Knowing where and quantifying forest extent is crucial for informed decision-making, enabling targeted efforts in climate change mitigation, biodiversity conservation, and sustainable land management. The below paper discusses the creation of a spatial database of planted forests, which include the genera (plural of genus) of the forests by combining remote sensing and on-the-ground data and feeding it to some ensemble models. (What a coincidence! A paper with a Purdue professor! I wonder why I put this in this article)

Relevant paper: https://www.nature.com/articles/s41597-023-02383-w?_ga=2.30766072.615500296.1702264608-866912038.1702264608

Crop Yield Prediction

Crop yield prediction is crucial for optimizing resource use, managing risks, ensuring food security, and supporting informed decision-making in agriculture and economic planning. You would expect that info like rainfall and temperature would be helpful. There are indexes like the normalized difference vegetation index (NDVI) which is basically this simple formula using the reflectance of light in an area to calculate vegetation health. However, ML algorithms can uncover their own “indexes” or rules to better predict yield. Like this paper below which predicts corn and soybean yield from the raw satellite data:

Outline of YieldNet architecture to predict corn + soybean yields from Fig. 1 of relevant paper below

Relevant paper: https://www.nature.com/articles/s41598-021-89779-z

Weather Prediction

I remember reading about this project from DeepMind when it came out about precipitation prediction: https://deepmind.google/discover/blog/nowcasting-the-next-hour-of-rain/

And recently I heard about their new GraphCast model that seems to have expanded capabilities like being able to predict cyclone movement: https://deepmind.google/discover/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/

I think what is most interesting about this paper is that they mention that the traditional weather models use physics equations. However, GraphCast does not have this built in. Perhaps the model may have came up with its approximated the real physics or came up with its own “pseudo physics” or maybe it just skipped it entirely. Yet, it still has high performance. Super interesting.

Useful Tools

Google Earth Engine

Description: Super super useful tool. Can access tons of up to date satellite imagery data like temperature, light reflectance, etc. You can also edit the data for your specific needs. Like for example stack 5 datasets together, get the average, only focus on certain country, get crop yield info, etc.

Link: https://earthengine.google.com/

QGIS

Description: An open-source Geographic Information System (GIS) software. It’s a user-friendly desktop application for viewing, editing, and analyzing spatial data. QGIS supports a wide range of data formats and offers various tools for mapping, geoprocessing, and creating interactive maps.

Link: https://qgis.org/en/site/

University of Texas Libraries Geo Data

Description: Really useful database of global country/state/county borders

Link: https://geodata.lib.utexas.edu/

Samgeo

Description: Tool to segment remote sensing imagery. Creators finetuned the Segment Anything Model (SAM) by Meta (https://segment-anything.com) which, given a prompt such as a point, bounding box, or text by the user, segments objects. They also have a “segment everything” mode where it spam 100s of input points, segmenting your whole image. Can be helpful if curating a custom dataset.

Segment everything mode done on aeiral image of a neighborhood. You can notice some parks, houses, and streets (the unlabeled white parts).

Link: https://samgeo.gishub.org/

Hope you were able to learn something new and/or cool!