Learning Notes from Planet Competition

Full name of the competition: Planet: Understanding the Amazon from Space — Use satellite data to track the human footprint in the Amazon rain forest

Image Credit: Alexey Noskov from this discussion thread

I started participating in this competition pretty late, essentially not until the last week. As a result I gave up using tiff images (instead using the pre-processed jpeg images), and focused on practicing building a image classification pipeline.

It was quite frustrating that my result keeps falling short of beating the best public scripts (I did get on par, and even a little better with the scripts in the end). The biggest problem I cannot figure out is when I use higher resolution (256x256 and 224x224) images, the increase in training time does not accompany with a increase in accuracy. In the end, my best submission comes from a model based on 128x128 images. It’s weird because the pre-trained model I used (VGG16 and Resnet50) should work better with 224x224 images. My guess is that I didn’t manage the learning rate well. I’ll update the post if later I somehow figured it out.

The two highlights of the learning experiences are:

1. Bcolz Array Iterator: I used the implementation by the wonderful Jeremy Howard. My RAM cannot fit all the 224x224 training/testing images, so I have to either read and resize images batch-by-batch, or store the resized images inside a bcolz.carray and read them when I need them.
2. Cyclical Learning Rate: The implementation I used also comes from Jeremy Howard’s class, written by Brad Kenstler. It’s a really simple idea to change learning rate at every batch, and in Keras it’s really simple to do. The implementation comes with three built-in modes: triangle, traingle2, exp_range. I tried all of them and used exp_range with gamma=0.9999 in my final submission.

I plan to update this post periodically until I finished reviewing this competition.

Update on 2018/01/23: I’ve found the cause of under-performance a while ago. It was a stupid bug which led to only a small portion of the parameters were updated at each step. Using JPEG files is actually enough to achieve top-level performance. I had also done some more experiments on this dataset and planed to write a independent post on the results, but I never find the time to write it.