Estimating Tree Damages Using Remote Sensing Data
An investigation using ensemble learning for the classification of tree damage classes to determine forest condition using the Black Forest as an example.
Approximately 30% of the area of Germany is covered by forest. Recent studies in Germany reveal that 5% less forest area exists than previously assumed. This has a significant impact on climate protection. It was previously assumed that 51 million tons of CO2 can be bound by trees. 5% less forest area means that almost 2.5 million tons of CO2 cannot be bound, which thus remain in the atmosphere and further damage the climate.
In addition, forest areas are of considerable importance for tourism and in this manner for the local economy. Measured by the more than 15 million overnight stays by tourists in 2021, the Black Forest is the second most visited region in Germany. With the increase in the death of trees, essential functions of the forest are lost. The basic functions include:
- reduction of pollution from harmful gases, dusts or aerosols in the air
- preventing runoff or the formation of cold air that threatens small plants, but also protecting settlements from sudden temperature changes and lowering heating costs in surrounding cities.
- support for soil and water conservation in the region
- preservation of the landscape and related to it, the forest serves as a recreational area for many city residents who spend their free time there
In Germany, forest health is measured each year in a survey based on needle and leaf loss through a sample inventory of trees. These inventories are done by hand: 2 experts are visiting the observation sites and taking the measurements. Trees are assigned to one of a total of five damage classes, with class 0 representing healthy trees and 4 representing dead trees. This is extremely cost and time intensive. In addition, only statistical methods can be used to determine how other trees are affected by damage. Due to this, in this article I investigate how remote sensing data can be used to conduct forest inventories to estimate the forest condition in the Black Forest.
The potentials of an automated forest inventory using remote sensing data include:
- significant cost and time savings when terrestrial forest inventories are no longer necessary.
- accurate determination of areas that are affected by damage and need to be treated by targeted measures. Cutting dead trees to protect the environment. Distinguish healthy from damaged and dead areas in the process.
- Continuous forest inventory: conduct forest inventories more frequently per season in order to better model the trend. For example, if long-lasting droughts occur, damage can be assessed immediately afterwards
Relevant data for this analysis
This study uses a variety of data from a wide range of sources.
The 2 most important data include remote sensing data from the Sentinel 2 satellites downloaded from Planetary Computer by Microsoft [1] and the labeled tree damage classes of the forest inventories provided by the Thuenen Institute [2] and the Forest Research Institute Baden-Wuerttemberg.
Remote sensing data
Remote sensing data from the Sentinel 2 satellites provide new data every 5 days. Thereby 13 spectral bands with a spatial resolution of up to 10 meters are available. However, clouds block a clear view of the Earth’s surface and should be considered outliers and clipped of the data. The European Space Agency (ESA) provides with the data a Scene Classification Layer which is ideally suited for the removal of clouds.
Various indices are available to specify vegetation performance based on remote sensing data. The best known is the Normalized Difference Vegetation Index (NDVI) [3] followed by the Enhanced Vegetation Index 2 (EVI2) [4].
The EVI2 is more sensitive to changes than the NDVI, especially in heavily vegetated areas, due to the weighting of the Near-Infrared (NIR). For the scene, the vegetation indices are as follows (clouds have been removed). It can be seen that the EVI2 data (right) varies more than the NDVI data due to sensitivity and therefore appears in a lighter green.
Labeled forest inventory data
Forest survey data includes various data measurement points distributed throughout the forest area. One observation station is thereby divided into 4 subareas. These are located in the 4 cardinal directions 25 meters from the center, and include at least 6 trees each. For an observation point, however, only the coordinates of the center point are available. The other coordinates must be recalculated from this information. For this purpose, the size of the trees is calculated via the Stand Density Index [5], which is decisive for cutting out the corresponding areas of the remote sensing data. Due to the fact that the trees cannot be uniquely identified, the median of a subsection is applied as the target class in each case and used for training the data. This ensures that individual dead trees, for example, are not a factor in what would otherwise be considered a perfectly healthy area.
Further data sources
In addition to remote sensing data and inventory data, other data sources like elevation data, ground information maps, and weather records are consulted.
Soil information maps of the Federal Institute for Geosciences and Natural Resources [6] contain a variety of information at the German or European level. These include e.g. the possible rooting depth, water quality or soil properties with respect to humus and / or stones.
In general “Digital Elevation Model” (DEM) and “Digital Surface Model” (DSM) are used, among other data, for landscape modeling. ESA provides such a model with a resolution of 30 meters for download [7]. This elevation information can be used to calculate slope inclinations as well as slope orientations, which can be decisive for the resistance of trees in the event of heavy rain or winds, for example. In addition, on north-facing slopes, solar radiation is different compared to trees located on the south-facing side of the slope.
Weather is considered to be the determining factor for vegetation. Prolonged droughts damage vegetation similarly to storms or heavy rains that can erode the soil. For this reason, daily weather records are downloaded from the weather archive and various attributes are formed. These include rain-free periods in summer, number of days with a particularly high temperature but also sunshine hours in spring or snow masses in winter in the corresponding period before the inventory.
Identification of similar location-based observations
The site-specific data consisting of the soil information maps, as well as the DEM information are examined for commonalities.
For this purpose, observation points with similar topography are divided into 14 clusters using the Gaussian mixture model. An examination with the DBScan results in a number of observations that cannot be assigned and are classified as outliers. These are not useful in identifying similarities, since the defined clusters are to be applied to the area of the entire Black Forest.
For this reason, the 14 clusters recognized by the Gaussian mixture model are used and are nominally represented by the one-hot encoding.
The data and generated attributes mentioned in this section have to be combined. In this case, the coordinates of the remote sensing data provide the target raster. The damage classes of the forest inventories determined by the median serve as the target class to be modeled later.
Modeling
For modeling, the multi-class classification problem (consisting of classes 0 to 4) is transformed into several binary classification problems. These binary classes include e.g. the positive class “healthy” and the merged class “not healthy” which consists of the damaged and dead instances accordingly. The same applies to the other classes whereby the damaged classes (1 = minor damage to 3 = severe damage) are considered once combined and once separated from the healthy and dead classes as a binary classification problem.
In addition, a large number of models using Python’s scikit learn and imblearn packages are formed, optimized and tested as part of the modeling process.
- Decision Tree Classifier
- K-Nearest-Neighbors Classifier
- Random Forest
- Balanced Random Forest Classifier
- Support Vector Machine
- AdaBoost Classifier
- RUSBoost Classifier
- Hist Gradient Boosting Classifier
- XGBoost Classifier
A hyper parameter tuning is performed for all models using the Stratified K-Folds cross-validator while randomly fitting parameter combinations for 5 folds each 100 times after identifying the 50% most relevant features for each model. In addition, for all models, the decision threshold is optimized using the ROC curve above which an estimate is seen as positive. The change in estimates using a confusion matrix can be seen in the following figure.
The above steps are run through 2 times for each class. Once with the observed distribution of the classes and a second time where the classes are balanced. For this, the SMOTE-ENN (Edited Nearest Neighbor) is used to combine the generation of synthetic data with the subsequent removal of instances that would have been classified differently due to the 6 direct neighbors.
Due to poor tree vitality conditions in some cases, the data are heavily skewed to the disadvantage of healthy and dead trees. For this reason, Cohen’s Kappa is used in addition to the F1 value, which includes the probability of occurrence of the target classes in the results. Random TP events are thus put into perspective.
The result of this modeling phase is a list of fitted models that are used as base models for the next stage. Selecting the 50% most relevant attributes in combination with both balanced and unbalanced data sets ensures that there is variance in the estimation results.
These various base models will produce the final estimate by forming ensembles into meta-models. Therefor, based on the individual base-model results, an order per model is established for each classification problem. The order is determined by the main metric Cohen’s Kappa. From the best performing models, a simple weighted majority vote is used to generate the final estimate. The determined Cohen’s Kappa is used as the weight for a model.
Results
The results of the meta learners generated from the base learners can be seen in the table above. It can be observed that for some classification problems there is a strong improvement of the results while other classes hardly show any change. Basically, considering the possibility of overfitting, the meta learner is used in each case. At the same time, a bifurcation of the results can be observed, with the healthy, damaged, and dead classes achieving significantly better results than the low to severely damaged subclasses.
The poor results for the three subdivisions of the classes low damage, medium damage and high damage lead to the conclusion that a subdivision is not possible. Various explanations are possible for this:
In general, a deterioration of forest conditions can be observed, but no continuous decrease in the vitality of individual trees. Rather, a variation between the damage classes can be observed. At the same time, a clear identification of the trees in the remote sensing data is not possible due to the spatial resolution of 10 meters. In close reference to this aspect, the remote sensing data is a view from above. The different light spectra are reflected by the first upper layers of the forest and recorded by the sensors. Here, there is no way to tell if the tree is affected by needle or leaf loss in the thicket of the tree. If a branch breakage occurs due to snow masses or storms, the satellites can register the light reflections of the still existing leaf / needle layers. A forester can detect needle and leaf loss much more accurately by walking around the trees during a terrestrial forest inspection and assessment and evaluate accordingly. For example, a loss of needles or leaves by five percentage points may result in a gradation of damage level that may not be detected by the remote sensing data in the models if the vegetation performance is assessed consistently.
Final Results
Merging the estimation results (without regard to the subdivision of the damaged classes) into a vitality state yields the following metrics:
- unweighted averaged F1 value: 0.701 with a Cohen’s Kappa value of 0.554 for conifers, which can be rated as moderate.
- unweighted average F1 value: 0.828 with a Cohen’s Kappa value of 0.654 for deciduous trees, which is to be evaluated as substantial
- unweighted averaged F1 value: 0.745 and a Cohen’s Kappa value of 0.577 for trees in mixed forests
With the results obtained in the modeling for the classes “healthy”, “damaged” and “dead”, a Cohen’s Kappa score of 0.582 and an F1 measure of 0.736 weighted from the coniferous and deciduous trees is obtained.
“Explainable AI” — Most important variables
To understand the models, the features considered for the meta-learners will be considered. These can be seen in the following figure.
A distinction is made between the different groups (vegetation, weather and location-features) and the target classes.
Overall, it can be observed that the ensemble learning and the selection of 50% of the variables per target class and model leads to a balanced distribution of attribute consideration.
Thereby, it can be noted that, in principle, the weather variables are used most frequently. At the same time, the location variables are least relevant for the detection of dead trees of all target classes. Here, the assumption can be made that the vegetation indices together with weather variables are sufficient to estimate the damage class “dead”.
When considering vegetation indices, it can be observed that both NDVI and EVI2 attributes are used. A separation can be seen, that for conifers EVI2 variables are used more often while for deciduous and mixed forests the NDVI attributes are used more often in absolute terms. These findings are confirmed by the relative frequency density per variable group.
Damage class prediction of spatial data
The image below shows an aerial view of the Black Forest taken on May 9, 2021. Next to it, the maximum NDVI value per coordinate measured in 2021 can be seen.
Considering the estimation (lower image), it can be seen that the largest area is classified as damaged and isolated areas are detected in the states healthy and dead.
For the prediction of the data, the tree distributions (coniferous vs. deciduous and mixed forests in case of overlap) have been considered.
Further Research
Analysis of the models confirms that weather variables have significant relevance on tree vitality. Further research could incorporate specific weather models into the modeling and look at effects of weather on vegetation from a new perspective to estimate forest health.
Impacts due to long-term droughts, such as an increasing lack of water that may be more frequent and severe due to climate change, can be correlated along with changes in vegetation indices and better represented by the models.
In addition, storms or snow masses, for example, which cause branch breakage and contribute significantly to needle and leaf loss, can be better represented. Findings in different tree species can be positively reinforced by the differentiated view and modeling of coniferous and deciduous trees.
Until new generations of satellites are available for documenting changes in the earth’s surface, which allow a higher resolution compared to the currently usable resolution of ten meters, measures are conceivable that can also be used in a traceable manner for the entire observation data of the inventory. A determination of the coordinates of the trees contained in the observation points could be applied retrospectively for all years until the respective trees were included in the sample size and lead to an improvement of the accuracy of the target classes.
Considering the different tree distributions of the Black Forest compared to the other forest areas in Germany, further modeling is necessary to make the models applicable for estimating the forest condition beyond the boundaries of the Black Forest. Due to the fact that the proportion of spruce is lower in the whole federal territory and the proportion of pine is significantly higher compared to the Black Forest the fitted models can not be adapted without any modification of weights. For this, the generated models should be considered for a transfer learning for further areas like the Swabian Alb, the Spessart, Taunus etc..
Following on from this, remote sensing data can be used to lead to significant improvements in the quality of forest inventories at an international level in similar forest areas across national borders. In turn, these data are crucial for statements regarding the development of the climate and global warming if trees are unable to bind CO2 as previously assumed due to damage.
Based on this, further measures to protect the forests or regulatory measures to reduce the climate-damaging gas by politics may be necessary.
Resources and Links
[1] Microsoft’s Planetary Computer: https://planetarycomputer.microsoft.com/dataset/sentinel-2-l2a
[2] Thünen-Institut: https://www.thuenen.de/en/
[3] NDVI: https://doi.org/10.1016/0034-4257(79)90013-0
[4] EVI2: https://doi.org/10.1016/j.rse.2008.06.006
[5] SDI: https://www.srs.fs.usda.gov/pubs/gtr/gtr_srs175/gtr_srs175_577.pdf
[6] Soil Data: https://produktcenter.bgr.de/terraCatalog/Start.do
[7] DEM: https://registry.opendata.aws/sentinel-2/
[8] Weather: https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/daily/kl/historical/
