Collaborative Seagrass Mapping from Sentinel-2 Satellite Imagery
Authors: Rachel Keay (UKHO), Izzy Hassall (JNCC), Clare Fitzsimmons (Newcastle University), Lewis Castle (JNCC)
This blog will share the recent investigation that the UK Hydrographic Office (UKHO) Data Science, Bathymetry and Remote Sensing teams have carried out to learn more about mapping seagrass from satellite imagery. As part of this work the UKHO collaborated with government partners, Joint Nature Conservation Committee (JNCC), and Newcastle University for essential ground truth data of seagrass locations over the British Virgin Islands (BVI), Caribbean.
What is seagrass & why is it important to map?
Seagrass is an underwater flowering plant that grows in marine and estuarine environments on all the world’s continents except Antarctica. It typically grows on flat soft substrates, such as sand or mud, with coverage ranging from sparse to very dense meadows of seagrass.
Marine habitats of seagrass stabilise the coastline, provide protection from storms, support fisheries and serve as a bioindicator of healthy oceans and coastlines. It is estimated that seagrass beds capture 10% of the world’s carbon dioxide despite occupying less than 0.2% of the ocean. However, seagrass is at risk from pollution, anchor damage, coastal and port development, and environmental change, such as increasing sea surface temperatures.
Introducing UKHO, JNCC & Newcastle University
The UKHO survey, process and validate a large variety and volume of data to create navigational products and data sets for maritime situational awareness and to support the development of sustainable marine economies. The UKHO is the Marine Environmental Data and Information Network (MEDIN) accredited national Data Archive Centre (DAC) for bathymetric surveys. Bathymetry describes the measurement of the depth of water and is essential for seabed habitat mapping and, therefore, useful for decision-making when it comes to sustainable development of our marine and coastal environments in the UK and overseas.
JNCC work closely with each of the devolved governments and country nature conservation bodies to develop a strong environmental evidence base for the UK to address obligations arising from international treaties and conventions. This is done specifically through data collation and asset management. JNCC also hold a collaborative partnership with a European consortium of government agencies and research institutions (called EMODnet Seabed Habitats) to provide a library of marine habitat maps, survey sampling points and modelled habitats for Europe. Using this data repository, regional composite data products can be created, including: seagrass cover as an Essential Ocean Variable and Zostera beds included in the list of OSPAR threatened and/or declining habitats.
Newcastle University has been at the forefront of tropical coastal management research and education for four decades. With an established history of work covering the full range of Caribbean marine environments, researchers conduct interdisciplinary work in support of fisheries management, habitat conservation, species protection and environmental monitoring and assessment, in the context of climate change. Recent projects have developed new methods for drone and satellite mapping of critical habitats, such as coral reefs and seagrass beds, and ways of integrating ecological knowledge and multi-resolution data sets to improve information on benthic habitats for Marine Protected Area planning.
Our aim & challenges
Our aim is to experiment with data science tools and techniques for supervised classification to detect and map seagrass from Sentinel-2 satellite imagery in the British Virgin Islands (BVI).
Finding seagrass in satellite imagery presents several challenges:
- Seagrass is underwater, so wavelengths of light from the sun must travel through the atmosphere and water column to reach the seabed before being reflected and detected by a satellite. Light interacts with particles causing scattering and absorption so the reflectance values provided by satellite data are not 100% representative of the seabed.
- Seagrass is not homogeneous; it grows in very sparse to dense meadows and is often intermixed with other substrates (e.g. coral, macroalgae, sand).
- There is a lack of in-situ observations (ground truth) and expertly labelled data to train a machine learning classifier for marine habitat classes. This labelled data is essential for training supervised machine learning models.
This blog describes how we were able to overcome some of these challenges and collaborate with colleagues from JNCC and the University of Newcastle.
Mapping British Virgin Islands seagrass
BVI is a UK Overseas Territory in the Caribbean and the water is often clear, making mapping marine habitats from satellite imagery feasible. We picked 6 areas of interest (AOI’s) that had 100% coverage of Sentinel-2 satellite imagery, bathymetry data, and a good mix of JNCC/Newcastle University marine habitat ground truth data.
The importance of ground truth data
To fully understand what the habitat cover is at a given location, we need accurate ground truth data. This data allows us to relate satellite image pixel values to what’s on the ground and is required to validate machine learning models. Confirming that model predictions accurately reflect habitat cover is vital in proving that the outputs are robust and meaningful. Collecting field survey data is very costly and resource-intensive, and there are important factors to consider. One major consideration is the timing of surveys — ground truth data need to be collected close to the satellite image date, making sure that the pixel values match the real habitat cover. For this project we used ground truth data collected by Newcastle University in June 2019 at 42 locations across the BVI archipelago. Underwater photographs were taken along transects and the broad habitat type was classified according to the dominant cover. For further details see the paper Investigating the impacts of the 2017 Hurricane season on the shallow marine environment of the British Virgin Islands using multispectral satellite imagery.
Image and bathymetry data
The European Space Agency (ESA) Copernicus Sentinel-2 mission collects satellite imagery with a 10-metre resolution in the visible bands and has a high revisit frequency of ~5 days. Level-1C products are georeferenced to UTM (Universal Transverse Mercator) projection and radiometrically corrected to top-of-atmosphere reflectances.
Bathymetric data is collected by LiDAR systems that use rapidly pulsing lasers from an aircraft. The LiDAR data for BVI, acquired from the UKHO bathymetry team, is cleaned to International Hydrographic Office S-44 standards, and formatted to 4-metre resolution geotiffs. This bathymetric data is useful for: (i) masking deeper water pixels and (ii) generating a marine geomorphology dataset (iii) as an input to machine learning models. Classifying geomorphology helps to inform habitat modelling by providing the seagrass classification workflow with the underlying physical structures of the seabed.
How we prepared the image and bathymetry data for machine learning
It is important to remove the effects of scattering and absorption in the atmosphere and water column so that the spectral reflectance value that our machine learning model learns from is representative of the seabed. Image data pre-processing included:
- Atmospheric and glint correction using the Royal Belgian Institute of Natural Sciences (RBINS) ACOLITE (version 20190326.0), see blog for more details.
- Water column correction is performed using Lyzenga’s algorithm, a popular empirical approach to correct the effects of light attenuating in the water column. Lyzenga’s algorithm calculates the attenuation coefficient ratio of two spectral bands with a regression line that minimises the mean square deviation using manually selected training pixels from ‘invariant’ areas (e.g. sand) at multiple depths. The attenuation ratio regression model is applied across a full image and each y-intercept is the index of a substrate type. The output is a depth invariant indices image.
Bathymetry data pre-processing: The data is re-sampled using GDAL to the same resolution as the satellite imagery. Geomorphologic data is created using the geomorphon classification technique developed in python by UKHO Data Scientist Michael Hudgell to create a data set with 10 classes: peak, ridge, shoulder, spur, slope, footslope, flat, hollow, valley, pit.
Creating labelled data
The JNCC/Newcastle ground truth data is not to be used to train any models, so it can be used as an independent, unseen test set with which to assess accuracy. To create labelled data points to train the model, locations are manually digitised. To support decision making we referenced Google Earth imagery, the Ocean Biodiversity Information System (OBIS), Seagrass Spotter and a myriad of tourism, yachting and diving websites telling us where to find seagrass in BVI. In total we ended up with ~4,000 seagrass points and ~8,000 not seagrass points for the 5 training AOIs.
The UKHO Remote Sensing team verified the labels. It is difficult to manually identify what is seagrass and not-seagrass because on visual inspection seagrass at 5 metre depths can look like coral at 15 metre depths. So, the expert eye and knowledge of the UKHO Remote Sensing team is valuable to make sure that we get our labelled classes right.
Machine learning
The data going into the machine learning models contained the RGB bands from Sentinel-2, bathymetry data, green/blue depth invariant indices, and geomorphology data. Because of the non-linear relationship between these features, three models are selected that are known to perform well for satellite image pixel-wise classification (aka semantic segmentation):
- Random forest; an ensemble of decision trees.
- Support vector machine (SVM) with a radial basis function (RBF); transforming data into a higher dimensional space.
- A convolutional neural network (CNN) called U-Net was originally developed for biomedical imaging but transfers across to satellite imagery very well. The U-Net approach for seagrass mapping does not use fully annotated labelled images due to uncertainty in labelling every pixel as either seagrass or not seagrass. Instead, a weakly-supervised U-Net approach is used that has been adopted from academic work and developed by two data scientists at the UKHO; Thomas Redfern and Pascal Philipp.
Methodology
Train, Validate, Test: To train a robust seagrass model we must make sure that it generalises well, therefore we train and validate on 5 of the areas of interest (AOI), and then test the model on a hold-out AOI for model evaluation. We then use the JNCC/Newcastle ground truth data points at the test AOI for final model evaluation in the test area.
Balance Data: To address class imbalance, the underrepresented seagrass class is up-sampled and the over-represented not-seagrass class down-sampled. The data for the U-Net model is balanced with augmentation (flips and rotations) on chips that contain mostly seagrass labels. The aim is to show a model a balance of examples, so it is not sensitive to the dominant not seagrass class.
Optimising Hyperparameters: To tune the hyperparameters for random forest and SVM we perform k-fold cross validation. U-Net hyperparameter tuning experiments are visualised and tracked with Tensorflow’s Tensorboard to evaluate the loss function on the validation data and make sure that the model is not overfitting while minimising the loss.
Model Training: Random forest and SVM were trained using a python library called Rapidsai CUML that mirrors python library scikit-learn. CUML is a machine learning library that uses the GPU resulting in improved computation time. While the U-Net model was trained using Tensorflow and Keras on the GPU.
Accuracy Assessment: Metrics are required that best quantify the accuracy of a model. Segmentation models are usually evaluated with:
- Mean Intersection-Over-Union (Mean IoU), which is a method to quantify the percent overlap between the target/true labels and our predicted output for each class then averages them. Also known as the Jaccard Similarity Index.
- Recall scores express a model’s sensitivity to seagrass instances.
- Precision scores express the proportion of predicted seagrass that is actually seagrass i.e. the true positive rate.
- F1 score is the harmonic mean of the recall and precision scores, also known as the Sørensen–Dice coefficient.
And visual interpretation (qualitative assessment) of a model’s predictions is also considered important to assess the detectability within the local landscape.
Results
The predictions from each of the three models are compared against the JNCC/Newcastle ground truth data points in the test AOI.
The random forest and SVM models have high precision and lower recall scores, indicating that these models are as not sensitive to the seagrass class and will often miss a seagrass pixel and over-predict the not-seagrass class. The U-Net seems to have a well balanced recall and precision score, meaning that augmentation efforts trained a balanced model that got almost as many seagrass predictions as not-seagrass predictions correct. For all three models the F1 scores between 0.7 to 0.79, suggesting that overall the models performed well to good, and the mean IoU scores between 0.53 to 0.66 with anything over 0.5 considered a good prediction.
Discovering what the model did not learn by visual (qualitative) assessment will help us to make improvements for future work.
Visual assessment suggests that all models:
- Performed poorly in locations where seagrass meadows are sparse. The SVM appearing to learn a little more than the random forest and weakly supervised U-Net.
- Performed very well over dense seagrass locations, with the weakly supervised U-Net showing well defined edges in particular.
- Struggled with false positives over coral intermixed with bare rock, mostly Gorgonian and massive corals. These corals can appear green and are spectrally easily confused with seagrass.
- The SVM and random forest seems to have overfit to ridges/slopes from the geomorphology layer that is intended to teach the model that seagrass grows mostly on flat substrates, while coral can grow on slopes and forms ridges.
Discussion and recommendations
We had many challenges to overcome for this work with many great learning opportunities. By experimenting with atmospheric, glint and water column correction we aimed to transform the Sentinel-2 imagery to show the true reflectance of the seabed. The cloud-free Sentinel-2 image selected for this study had high levels of sun glint due to high winds. Sun glint correction should improve the imagery, but noise can be introduced and therefore negatively impact the data going into model training. A couple of approaches to improve input imagery would be: (i) find glint free (or as close to glint free) imagery as possible or (ii) create a composite image with median pixel averaging techniques.
Classification of dense seagrass was more successful than sparse meadows. The model was not shown enough sparse seagrass labels to learn from because they were difficult to identify when manually labelling. And, the sparse seagrass class is more difficult to predict because those pixels often contain a mix of seagrass and coral, sand, and/or macroalgae. Therefore, it would be interesting to experiment with separate classes for seagrass densities (low, med, high) using a multiclass model or two separate seagrass models (dense and sparse) to specialise the model and give the model a better chance to learn sparse seagrass spectral patterns.
The model could be improved with more labelled data. To overcome shortages of labelled data I would recommend more acquisition of in-situ seagrass data, data discoverability and collaboration. We have shown that collaborating with other organisations and academia offers data and technical knowledge sharing.
A big thank you to everyone who has contributed to this work in defining methodology and developing the skills for marine habitat mapping work.
