Published in


Dynamic Time Warping for Satellite Image Time Series Classification

A tale of pastoralism, land degradation and data scarcity

Did someone say TIME WARPING?!

My girlfriend often asks me what I am working on at that particular moment, and I very often have no interesting answer to provide to her… Until I started working on… DYNAMIC TIME WARPING!!!!

My favorite time-lapse ever made from satellite imagery, drawing out the analemma pattern in the sky. Credits go to: Matic Lubej
  • Matching heterogeneous time series: Time series of varying lengths and sampling intervals are the norm in multi-temporal satellite imagery processing. Temporally aggregating time series of images to fewer composite products corresponding to a given temporal interval (e.g. 15 or 30 days) is common to harmonize time series for methods that require it, which is most machine learning classification techniques. On top of time series harmonization, temporal aggregation wields the benefit of data size reduction, noise reduction, and gap-filling (e.g. cloud gaps in optical data). However, even if you harmonize your image time series, there is no guarantee the available reference data is sampled at corresponding time intervals, unless the reference data pattern is interpolated to match the time series sampling, which would create synthesized values that may not correspond to reality. Instead, DTW can directly handle the mismatch between the pattern length and the image time series length!
  • Handling of pseudo-periodic phenomena: When talking about vegetation, the modulations of phenological cycles result in distortions of their canonical temporal profiles, both in space and time. In time when dealing with inter-year climatic variations of vegetation response, and in space when dealing with the delayed start of the growing season along a north-south gradient, or with varying altitude. Allow me to paraphrase Petitjean et al., 2012 who describes this issue as the intrinsic temporal behavior of the observed object, as opposed to the temporal irregularity resulting from the sensor’s frequency of revisit (and cloud presence/absence for optical payloads), described in the previous bullet.
Illustration of pattern matching using DTW as opposed to the rigid Euclidian distance. Taken from Databricks’ excellent explanation of how DTW works

The Constraint

The advantages of DTW are clear, especially in a data-scarce context where flexibility is required to manage heterogeneity of the data sources available. However, too much flexibility in the pattern matching behavior can lead to poor performance. Indeed, by comparing the full temporal range when finding the best alignment between two time series, DTW excludes contextual knowledge about vegetation seasonality and phenology, whereas it could be integrated as knowledge to the system.

The alignment between two sequences is computed only for the yellow cells of the matrix, reducing the number of computations necessary. Taken from Csillik et al., 2018
Linear and logistic time-weight. The logistic weight in this case has midpoint β = 100 days and steepness α = 0.1. Taken from Maus et al., 2016

A Land Cover Classification Use Case for Sudan

To provide a bit of concreteness to this theoretical blabber, we are applying DTW with SoilWatch as part of an effort to design, plan and undertake climate-financed forest and landscape restoration at scale. Bear with me for this context intermezzo, where I try to provide the why and the how.

No longer good for annual crops, no longer good for perennial grasses: Mechanized Agriculture in the Butana Communal Rangelands have turned the area into a desolate wasteland. Courtesy of: SoilWatch
Degraded rangelands in Darfur being ploughed for a few years of opportunistic sorghum farming, destroying the little that is left of the perennial grasses’ root systems holding the soil together, resulting in barren land void of any life forms. Courtesy of: SoilWatch
  • Are we in a situation of low contextual/local knowledge availability? ✔️

Bringing the algorithm to the data

I wanted the approach to be scalable, so decided to bring DTW closer to the data by implementing it in Google Earth Engine (GEE). After some years of the algorithm being requested on the GEE developer forum, GEE users will be delighted to find the code here. It allows multi-dimensional inputs, meaning that multiple spectral bands or indices can be provided across multiple sensor types. In this example, I have used 7 bands: B2, B3, B11, B12, NDVI from Sentinel-2, and VV, VH from Sentinel-1. The former provide critical phenological information, while the latter provide structural information about the surfaces observed. They are combined through a simple Euclidean distance, as in Csillik et al., 2018:

For combining multi-band DTW outputs, the Euclidean distance is used to compare the elements of time series where aif and bjf are elements of the feature f of multi-dimensional sequences A and B, with F number of features.
  • Water.
  • Trees, combining both natural and planted.
  • Wetland.
  • Bare Soil. This may be rocky outcrops, or degraded land to the extent where nothing grows back (see picture below).
  • Rangelands. They cannot be effectively separated from abandoned or long-term fallowed land based on a single-year DTW output, but can be differentiated by using previous years’ outputs to identify a crop phenology prior to fallowing.
  • Active Cropland in 2020, whether rainfed or irrigated.
  • Abandoned/long-term fallowed cropland in 2020. A field is considered abandoned if fallowed for more than 3 years in a row, using multi-temporal land cover output.
  • Short-term fallow cropland in 2020. Fallowed for less than 3 years in a row, using multi-temporal land cover output.
Serious soil erosion in West Darfur, where opportunistic grasses grow around bare soil patches. Courtesy of: SoilWatch
A crazy kaleidoscope of phenologies between July and December (one frame for each year from 2017 to 2020) in the Sennar District of Sudan. The false RGB composites was produced with the following Sentinel-2 bands: Red: December NDVI, G: October NDVI, B: August NDVI
The all-class combined dissimilarity score, ranging from 0 (green) to 20000 (red), is helpful to understand areas that are poorly represented in the reference dataset. This can guide the collection of new samples in those given locations, or the filtering of the land cover map using a dissimilarity score threshold to mask areas of low classification certainty.
The land cover classification using the Time-Weighted DTW approach. Most of rangelands and disputed croplands (green: rangelands, orange: short-term fallowed croplands and turquoise: long-term fallowed cropland) are located in the north and east of Sennar District. Our approach provides fine-grained (10m resolution, the native resolution of Sentinel-2) information about the distribution and status of rangelands and cropland across the area.
Land Cover Class Distribution in the Sennar District
Historical land use map for Sennar, as provided by Sennar Information Centre. Unfortunately, I was not able to find out the date of map production.


Dynamic Time Warping applied with a time weight is an incredibly flexible approach to satellite image time series classification, due to its ability to perform in data-scarce conditions — which, let’s face it, is often the case — , as well as its ability to deal with heterogeneous data sources across years.

  • Maus et al. 2016’s paper on time-weighted DTW applied to land cover classification
  • Ovidiu Csillik & Mariana Belgiu’s research on applying dynamic time warping to crop type classification:
  • Hussein M. Sulieman’s research in Gedaref state:
  • My consultancy work at FAO for TCP/AFG/3706 and lorenzo de simone have contributed a great deal in shaping the implementation of DTW in GEE, where it is applied in the context of crop type mapping.



Remote monitoring | Soil Regeneration

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
William Ouellette

Regenerating degraded 🌍 and eroded soils 🌱, one pixel at a time🛰️