Fighting Hunger through Open Satellite Data: A New State of the Art for Land Use Classification
Nowadays, we have access to a tremendous amount of high-resolution satellite images of the earth. And with tools like Google Earth Engine, getting the data has never been easier and faster.
This huge amount of open data is a great opportunity to use AI for Good and create useful applications.
For the past 4 weeks, I have been working with a team in one of Omdena’s AI Challenges in collaboration with the United Nations World Food Program.
The goal is to fight hunger in Nepal by locating, tracking, and improving the growth of crops such as rice and wheat.
Getting open data: the Sentinel program
To locate the crops in Nepal, we need an open and good source of high-resolution satellite images.
One of the best open data sources, in 2019, is provided by the Sentinel satellites of the Copernicus program (European Space Agency).
The satellites cover the full earth on 13 bands with a revisiting every 5 days.
The Sentinel 2 spatial resolution is 10 meters per pixel (and only for some bands). It is a much lower resolution than what we would get using paid services (some services provide images with a resolution of less than 1m per pixel).
Using this 10 meters resolution to learn crops classification or segmentation is quite a challenge, especially in Nepal where crops fields are not that big.
Thankfully, Deep Learning is a powerful tool we can use for this job.
Furthermore, because we don’t have a lot of labeled data on Nepal crops, it would be helpful to benefit from a pre-trained Neural Network. Starting to train a model from scratch would require much more labeled data.
Usually, in Computer Vision, we can get great results using models pre-trained on the ImageNet dataset (14M images in 20.000 categories). But the ImageNet categories (animals, plants, food, …) are very different from satellite imagery.
Can we do better for our use case?
Entering the EuroSAT Dataset
The EuroSAT dataset is composed of 27.000 images of European countries, labeled in 10 classes (crop, highway, industrial, river, …).
The images are from Sentinel-2 satellites:
- Image patches of size 64x64.
- Resolution of 10m / pixel for all the bands (bands with lower native spatial resolution have been upsampled).
The EuroSAT paper (fev 2019)¹ explains the dataset and how they got a SOTA result on the classification task.
Their best results are:
- a 96.43% accuracy with a ResNet-50 model trained from scratch.
- a 98.57% accuracy with a ResNet-50 model pre-trained on ImageNet.
They have compared the best combinations of 3 bands and have found that RGB (red green blue) works the best. So they used RGB images as input to their models.
But, they did not try to create a model using all 13 bands on a custom architecture. The other bands should provide more information to a Deep Learning model.
Of course, this would be quite different from a ResNet-50 pre-trained on ImageNet.
But what if we try?
Using multispectral images, we got a new SOTA result!
To use all the bands, we have to create a new model taking 13x64x64 images as input.
There are lots of ways to do that (it’s even possible to reuse a model pre-trained on Imagenet, adding a layer to reshape the 13 channels as 3 channels).
After several experiments, we got the best result using a ResNet like architecture (xresnet with the fastai library), starting the training from scratch.
In only 5 minutes of training (on a single consumer GPU), and without any data augmentation, the model has already beaten the SOTA result with a 98.69% accuracy. This is much better than the Resnet-50 trained from scratch and it even beat the model pre-trained on the full ImageNet dataset!
Using the MixUp data augmentation² and longer training, the best result we got is a 99.0% accuracy.
It’s a 30% reduction of the error rate compared to the best pre-trained model on ImageNet!
Using multispectral images is definitely helping!
Try it on your own applications
If you want to create classification models using the Sentinel 2 data, you can download our pre-trained model.
This GitHub repository gives:
- The weights of the pre-trained model and the code to use it.
- Notebooks to explain how to download Sentinel images using Google Earth Engine and how to process and visualize multispectral images.
With the power of open data, let’s make the world a better place!
If you want to receive updates on our AI Challenges, get expert interviews, and practical tips to boost your AI skills, subscribe to our monthly newsletter.
We are also on LinkedIn, Facebook, and Twitter.
Sources
[1] Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. Patrick Helber, Benjamin Bischke, Andreas Dengel, Damian Borth. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019
[2] mixup: Beyond Empirical Risk Minimization. Hongyi Zhang and Moustapha Cisse and Yann N. Dauphin and David Lopez-Paz. International Conference on Learning Representations, 2017.
