Deep Learning with SAR
In collaboration with ASTERRA
I recently caught up with Inon Sharony, the Head of AI at ASTERRA, to learn about the fascinating world of machine learning with SAR imagery, and you can view the full conversation below. Inspired by that conversation, this post covers some of the applications that utilise SAR, provides a short introduction to the essential considerations when using deep learning approaches with SAR data, identifies emerging trends in this area and provides resources for getting hands on with SAR & deep learning.
A very brief introduction to SAR
Synthetic Aperture Radar (SAR) systems use the reflection of radar signals, rather than light, to construct images. This provides SAR with the capacity for all-weather operation and the ability to capture imagery during both day and night conditions.
Beyond image acquisition, the polarimetry and phase information contained within SAR data enables polarimetric SAR (PolSAR) and interferometric SAR (InSAR). PolSAR is a technique that provides detailed information about the physical properties under the Earth’s surface, whilst InSAR can measure small displacements in the Earth’s surface.
The underlying physics of SAR and its contrast to optical imaging is a detailed topic, and to learn more I recommend viewing the Minds Behind Maps episode below (also available as an audio podcast):
Monitoring with SAR
The utilisation of SAR for routine monitoring tasks is extensive, and SAR may be used alone, or to augment optical imagery by providing additional information such as soil moisture levels.
Underground monitoring of critical civil infrastructures
PolSAR is sensitive to changes in moisture up to 3m deep. This enables the detection of changes even before their effects become visible at the Earth’s surface. This capability is utilised by ASTERRA in their EarthWorks solution for soil moisture monitoring and analysis in the presence of roads, railways, dams, levees, as seen on the ASTERRA website. PolSAR also enables the early detection of underground leaks from water and wastewater.
Infrastructure Development Tracking
The development and maintenance of infrastructure such as roads, bridges, and buildings, requires precise and timely information on a variety of factors, including land topography, structural integrity, and changes in the local environment. While inSAR is mostly known for detecting minor displacements in a series of multiple images, PolSAR is useful for identifying subsurface changes and soil moisture risks in just a few images — typically before any visible movement can be detected.
Deforestation and biomass monitoring
Many forests experience almost permanent levels of cloud cover, and SAR has emerged as a powerful modality for routine monitoring of these forests. SAR data, when processed with deep learning segmentation models, enables the generation of detailed deforestation and biomass maps.
The BioMassters dataset & challenge (featured in newsletter edition #3) tasked entrants to estimate the yearly biomass of different sections in Finland’s forests using imagery from Sentinel-1 and Sentinel-2. Checkout the winning solutions which utilised deep learning.
Ice and glacier mapping
SAR plays a vital role in the monitoring of ice and glaciers, particularly in regions of the Earth that experience semi-permanent darkness. SAR imagery can reveal changes in glacier volume, movement, and melting patterns, contributing vital data to studies on climate change and its effects on global sea levels and polar ecosystems.
Flood Mapping
During flood events, cloud cover and precipitation can limit the use of optical imagery. In this situation, SAR can provide data essential for mapping flooded areas. This information is typically used by emergency response teams to assess the extent of flooding, plan evacuation routes, and coordinate relief efforts. To learn more about this application I highly recommend the Microsoft blog post How to map floodwater from radar imagery using semantic segmentation
Preparing SAR Data for Deep Learning
SAR data’s rich, multidimensional nature demands a thoughtful approach for use in deep learning. The key is ensuring that models can effectively process SAR’s full data space, including phase and polarization, alongside intensity. For a comprehensive overview of this topic I recommend Deep Learning Meets SAR.
Critical aspects to SAR data preparation include data normalisation, speckle noise reduction, geometric and radiometric terrain correction, coherent polarimetric decomposition (using the Mueller matrices), and (in some circumstances) interferogram creation. Check the level of processing that has been applied to the SAR imagery you are using, but as a deep learning practitioner, you will typically need to perform data normalisation (with speckle noise reduction being a more subjective choice) regardless of the level of processing.
Data Normalisation
SAR backscatter intensity operates on a root-power scale, and when plotted, SAR data typically exhibits a Gamma distribution. Taking the log-transform results in an almost Gaussian distribution, now measured in decibels (dB). This transformation is crucial because many machine learning and optimisation algorithms are designed to perform optimally with Normally-distributed data. Normalisation in this way ensures smoother training and better convergence in model training and algorithmic performance.
Speckle Noise Reduction
Speckle is a pervasive form of noise in SAR images, which can impair a model’s ability to learn meaningful features. Employing speckle reduction techniques that balance noise mitigation with the preservation of image details is vital for preparing SAR data for deep learning models. Dedicated deep learning models have been developed to perform speckle removal (despeckling) and used during the data preparation phase.
Architectures
Deep learning models, particularly those based on CNN architectures, are inherently designed to learn hierarchical feature representations directly from the data, making them well-suited for extracting complex patterns in SAR imagery. Tailoring network architectures and loss functions to accommodate the specific properties of SAR data, such as the significance of phase information in interferometric applications or the handling of polarisation diversity, can enhance model performance. A review of the myriad custom architectures is beyond the scope of this article, but a comprehensive list is published on Github.
Emerging Trends in SAR and Deep Learning
The landscape of SAR technology and deep learning is rapidly evolving, with two notable trends poised to shape the future of satellite imagery analysis.
Integration with Other Modalities in Foundational Models
Foundational models have gained prominence in machine learning, particularly for scenarios where there’s an abundance of unlabeled data but only a limited number of labeled examples. These models leverage self-supervised learning techniques to extract meaningful patterns from extensive unlabeled datasets. Subsequently, they can be fine-tuned with a comparatively smaller set of labeled data to achieve specific tasks. This approach not only addresses the challenge of data scarcity in supervised learning but also enhances the model’s versatility and adaptability to diverse applications.
A significant emerging trend is the use of SAR in conjunction with other data modalities within foundational models. This multi-modal approach leverages the strengths of various sensor types, such as optical, infrared, and SAR, to enable a more comprehensive understanding of the observed environment. The recent addition of Sentinel 1 data to the Major TOM Core dataset is a notable example of the massive new datasets being created for the training multi-modal foundational models, such as the Clay Foundation Model featured in Newsletter #25
Expansion of SAR Data Products and Satellite Constellations
The proliferation of SAR data products and the expansion of satellite constellations mark another significant trend. As the demand for timely and reliable Earth observation data grows, so does the deployment of SAR satellites, both by governmental space agencies and private enterprises. This expansion is leading to a dramatic increase in the availability of SAR data, with higher temporal and spatial resolutions becoming the norm. The abundance of data is fuelling the development of new SAR-specific products and services, catering to a wide array of applications.
These trends, the integration of SAR with other data modalities in foundational models and the proliferation of SAR data products and satellites, are not only expanding the capabilities of SAR technology but also setting the stage for transformative changes in how we harness and interpret Earth observation data. As these trends continue to unfold, the synergy between SAR and deep learning will undoubtedly unlock unprecedented possibilities in satellite imagery analysis, driving forward innovation and offering deeper insights into our planet’s dynamics.
Get hands on with SAR
If you wish to get hands on with SAR data and deep learning models, I recommend investigating the datasets & pretrained model weights within TorchGeo. This open source project provides easy access to a number of SAR datasets:
- BioMassters Dataset for Aboveground Biomass prediction
- The SEN12MS dataset of Sentinel-1 dual-pol SAR data & Sentinel-2 multi-spectral images, with MODIS-derived land cover maps
- South Africa Crop Type Challenge dataset
There are also pre-trained weights for Sentinel 1 & 2 which provide an excellent foundation for fine tuning models on the datasets above:
Learn more about TorchGeo in the video below:
Conclusion
SAR has emerged as a key asset in addressing various complex issues. Its unique capabilities enable new applications not possible with traditional imaging approaches, and ASTERRA have developed innovative services which are particularly well suited to monitoring the urban environment. These enable the monitoring of critical infrastructure, and help their clients achieve their ESG targets set by the UN’s SDG plan for 2030 by better minimising the environmental impacts of climate change and ultimately saving lives.
The wealth of detail in SAR imagery presents a solid foundation for deep learning applications. Nevertheless, preparing SAR data for deep learning requires careful preprocessing, and there’s a growing effort to develop deep learning models tailored for SAR. Combining SAR’s robust sensing abilities and the analytical strengths of deep learning points towards a future where we not only manage SAR data’s intricacies, but actively utilise them to innovate and expand the scope of satellite imagery applications.
