AI-based Super resolution and change detection to enforce Sentinel-2 systematic usage

By Maximilien Houël, Leon Stärker and Stefano Natali

Since the launch of the first satellite (Sentinel-1A, April 1st 2014) The Copernicus Program has generated and provided an unprecedented amount of free and open data to monitor the Earth system as whole. Among the various satellites composing the space component, Sentinel-2 supports a wide range of applications, from agriculture to security passing through natural disasters, providing multispectral optical imagery at 10m resolution at global scale every five days. Unfortunately for such domains as security or for humanitarian response, the spatial resolution proposed remains too coarse for precise analysis and commercial very high resolution (VHR) satellites still have a financial barrier for many users. In the context of the European Space Agency (ESA) project «Earth Observation for Yemen» (EO4Yemen), artificial intelligence solutions were proposed to increase the operability Sentinel-2 and to support the humanitarian aids of specific areas over the country. Indeed the region is facing a catastrophic situation since 2015 because of an ongoing civil war, some areas are targets of airstrikes and military conflicts forcing the population to constantly move from areas to find safer places. In this context of war, satellites provide data over time to monitor the situation. A workflow was proposed to perform change detection over the country, in one way to identify the impact of war over cities but also to monitor the displacement of the refugees by using a proxy as the apparition of new settlements. This work aims at proposing innovations in remote sensing and deep learning fields but also to provide as much as possible precise information over the country by using Copernicus and open source datasets.

The full data analysis workflows foresees three steps:

1. Super Resolution : With the rise of AI applications, enhancement of satellite images is now feasible «artificially», in this context we proposed a solution to perform a super resolution over several Sentinel-2 MSIL1C bands (near-infrared, red, green and blue). For this purpose Worldview-2 images (2m) were used to create a reference dataset and increase the spatial resolution of the Copernicus sensor from 10m to 5m (Figure 1). The enhancement permits to answer multiple questions, indeed with a better geometrical definition of the buildings over the created images, smaller objects can be identified and be used as a proxy for population displacements.

Figure 1 : Super resolution example over the mosquee of Sanaa. [left : Sentinel-2 10m, right : Sentinel-2 SR 5m]

Figure 2: Super resolution example over the Louvre Museum in Paris. [left : Sentinel-2 10m, right : Sentinel-2 SR 5m]

Figure 3: Super resolution example Village around Marib City, Yemen. [left : Sentinel-2 10m, right : Sentinel-2 SR 5m]

Figure 4: Super resolution example Village around Sanaa City, Yemen. [left : Sentinel-2 10m, right : Sentinel-2 SR 5m]

2. Building classification : The generated super resolution images are then integrated into an AI classification to automatically identified buildings over areas of interest. The model was trained with open source building dataset over Africa regions with similar landscape than Yemen to be able to transpose it to the country we are interested in. This step consists in providing a preliminary analysis of an area by identifying automatically all the buildings on a Sentinel-2 image with a resolution of 5m (Figure 5). Small settlements can now be identified and with a temporal analysis evolution of the urban area is highlighting.

Figure 5: Building classification over the city of Sanaa

3. Object based change detection : Finally the preliminary analysis made over the buildings is refined by a change detection algorithm, focusing directly on objects. This algorithm performed a detection of changes as apparition of new settlements (Figure 6), but also disappearance of buildings identified in the past. This information can be used as proxy to monitor population displacement over several regions but also the impact of the conflict on the urban tissue. The method proposed is based on parameters directly linked to the classification and can be adapted with the type of landscape seen on the images (high urban density / isolated buildings …).

Figure 6: Change detection example over the city of Marib

The entire workflow, is in on one hand improving the use of the Open Source dataset provided through the Copernicus program, indeed this work is integrated into an innovation process with the use of artificial intelligence solution for earth observation analysis. On another hand it gives a meaningful support for humanitarian aids over specific areas where it is difficult to operate directly on the ground. The needed Sentinel-2 data are automatically collected through the ADAM platform (https://adamplatform.eu) APIs, that allow on-the-fly spatial and spectral subsetting. As further evolution, Super Resolution and Change Detection will be integrated within ADAM as processing functionalities, exploiting the elastic computational capability offered by the platform, to scale up at larger domains (.e.g. regional, national).