Mapping burned areas from space with Artificial Intelligence
Forest fires are a natural hazard often with dramatic impacts over ecosystems and human populations. Mapping the fire events accurately is very important for fire management, improving fire danger forecasts, monitoring air quality and fire emissions, and so on. Here I will briefly describe the main ideas behind the application of artificial intelligence to map burned areas using satellite images. For a more technical overview please refer to the article and the respective code implementation.
Can’t we map burned areas with satellite images already?
Traditionally the procedure to map burned areas using satellite images consists of carefully crafted algorithms. In the image below, for example, a burned scene is shown, obtained from Sentinel-2 satellites. The image on the left is similar to regular photography where the RGB colours are displayed. The image on the right includes instead the near-infrared band in place of the Green channel. The dark region in the right image identifies a burned area. A basic approach to map burned pixels could be as simple as defining a threshold value to indicate how dark the pixels should be in order to be mapped as burned. The current top approaches are of course much more refined and accurate than applying a simple threshold to a post-fire image. To give an example, a significant improvement to the method described previously can be achieved by considering images before and after the fire event and computing the differences.
Unfortunately, not every case is as clear as the one above or the problem would be quite simple. Moreover, the Sentinel-2 image shown features a resolution of 10 meters (i.e., each pixel has a 10x10-meter size), resulting in a huge volume of data for large scale applications. Here instead, 1km resolution images from Suomi NPP satellites are used. Those also have the advantage of new images being available every day, hopefully allowing not only to map the burned region but also to find on which day it has burned.
How can Artificial Intelligence be used to map burned areas?
The range of AI applications is very broad. The problem of mapping burned areas from satellite images is in the domain of Computer Vision and more specifically semantic segmentation — a task where the goal is to attribute a category to each pixel in the image. If ground truth or target labels are available, then the problem is referred to as supervised. Fortunately, for burned areas mapping, the existing products based on traditional methods can provide reasonably good targets. These products, however, are usually more accurate spatially than temporally — the assignment of the day of burning is often spatially incoherent, particularly if images are contaminated by clouds that make the task harder for any algorithm.
Instead of trying to write a complex algorithm to define which pixels have burned, in the AI approach, a model with a few millions of learnable parameters is defined with an architecture capable of learning spatial and temporal patterns in sequences of images. The model then looks to the images and the targets and tries to make the output as close to the target as possible.
Can the model predictions be better than the targets it’s trying to match?
The reader may now be asking, if the model tries to match as well as possible the existing targets then it can never be better than the existing products used as targets right? — Actually it can! Let me explain how. As I mentioned before, the existing targets are not as accurate in the assignment of dates of burning to the pixels. The AI model has, however, the capability to learn spatiotemporal relations in the data. Therefore, in order for the model to generate outputs that are consistent among all the training data, it cannot perfectly match the targets (unless it overfits, but there are several techniques to avoid that).
Not convinced? Consider this example. You are training a model to look at photos of cats and dogs and tell them apart. Imagine you have thousands of images and the respective targets but some of the targets are purposely mislabeled, i.e., you tell the cat picture has a dog or the opposite. After being trained, the model will probably classify these cases correctly (against the targets) since it is the only way the predictions can be consistent among the thousands of images, unless it memorizes every single image. Fortunately, there are several regularization techniques that can be applied to avoid overfitting.
But what is the “model”?
Without going into much detail, the model is a set of differentiable operations, including kernel convolutions in space and time, downscaling and upscaling layers, and an LSTM layer (a class of recurrent neural network) to capture longer temporal relations in the data. To the reader interested in learning more about practical deep learning, I recommend the fast.ai Practical Deep Learning for Coders course. It’s free and arguably the best deep learning course to get you started in a few weeks if you are already familiar with Python.
Training the model
Unless you are working on deep reinforcement learning applications the good news is that you don’t need to take the model to the gym in order to train it. But jokes apart, consider the following diagram:
The model — BA-Net — is given a set of 64 images of consecutive days for a 128x128 pixel region as inputs. It then generates a sequence of the same size indicating the probability that each pixel has burned on each day. (Note that in the figure above, the output and target represented are the results after post-processing, for illustrative purposes.) The output sequence is then compared with the target sequence using a Loss function. This loss function also needs to be differentiable so that gradients can be backpropagated to each layer of the model. In this Binary Cross-Entropy is used. The model parameters are then updated by a slight amount based on the gradients in order to walk in the direction that decreases the loss (i.e., increases the similarity between the output and the target). This process is repeated using a diverse set of input samples that allow the model to learn the spectral signature of a fire in the spatiotemporal data.
A case study
This case study refers to a fire that occurred on 3–9 August 2018 in Algarve, southern Portugal. This is a relatively easy case since almost no clouds are present in the input images. The figure below shows on the first row the sequence of daily input images from 3–9 August. On the second row the BA-Net near-real-time (NRT) product (i.e., at each day the model generated the outputs only based on the past 64-days). On the third row the BA-Net monthly product is shown (i.e., at each day the model is allowed to see days before and after). Finally, on the last row, the MCD64A1 Collection 6 (the product used as the target) is displayed for comparison.
As the figure above shows, the BA-Net NRT (second row) shows a similar pattern to BA-Net monthly product (third row) but with about three days of delay. On 12 August, three days after the reported end of the fire, BA-Net NRT already shows a burned map very close to the monthly product, showing the usefulness of this model for near real-time monitoring of burned areas. The difference between BA-Net (third row) and MCD64A1 (last row) is usually not larger than 1 day, however looking at the pattern of the fire BA-Net shows a clear starting point and continuous progression of the fire along the days, whereas MCD64A1 show two disconnected burned regions on 5 August that could suggest two fires merging on 6 August. It is known that this is not the case as the figure below shows where a reference high-resolution map obtained with human supervision is compared with BA-Net and MCD64A1 products.
It is interesting to note that there is apparently a 1-day delay for BA-Net compared to the Reference map, otherwise, the match would be remarkably close. However, there is a quite simple explanation for such a delay. As the BA-Net days are between two satellite observations and the period of larger fire expansion usually occurred after the time of the satellite passage, then the area burned during the afternoon hours is only seen on the following day — hence the 1-day difference.
Final remarks
The results obtained on this work are just a starting point for the application of deep learning to the burned areas mapping and dating problem. The rapid development in AI technology and computer processing power will continue to drive advances over the next years. I can’t wait to see what the next few years will bring!