Detection of aircraft over the Jorge Chavez International Airport in Lima, Peru with Mask-RCNN.

RarePlanes - An Introduction

Jake Shermeyer
Dec 10, 2019 · 4 min read

This past year CosmiQ decided to create a new dataset for object detection and instance segmentation of aircraft and their attributes in satellite imagery that we ultimately entitled “RarePlanes”. Throughout the dataset creation process we quickly learned that hand annotation of features in satellite imagery is difficult, expensive, time consuming, and potentially a mind-numbingly tedious task. After spending hours and hours labeling aircraft, correcting attributes, and listening to Radiohead’s most depressing songs well above the OSHA recommended decibel levels, I began to question my own existence. Why would I, like many other scientists and engineers the world over, subject myself to such dull and uninteresting work?

If you follow our work and this blog, the answer is probably unsurprising: to train data hungry computer vision algorithms! Supervised deep learning feeds off of the combination of human annotated labels and imagery, and typically the more well labeled data, the better the performance (at least for awhile). But what if there was a work-around to this laborious workflow? What if we could rely on an alternative method to generate labeled datasets? Thankfully there is an intriguing option that’s been recently introduced proposing the use of gaming engines to generate “synthetic” data. Synthetically generated datasets have been built for dozens of tasks, such as: aiding self driving cars, indoor 3D navigation, optical flow, and even robotic chefs.

Research has shown that such synthetic datasets can reduce the amount of real training data needed, and potentially improve performance for certain tasks. But geo is hard (Space Club Rule #36), and at CosmiQ, we don’t believe the hype until we test it ourselves. Consequently, we set out with the ambitious goal to build our own dataset to test out where synthetic data works, where it breaks, and provide data and methods to aid the development of a whole new set of vision tasks for the community to tackle.

Some chips from the real dataset (Maxar WorldView 3 Imagery) with attendant annotations outlined in red.

Today we introduce RarePlanes, a machine learning dataset and research study that examines the value of synthetic data to aid computer vision algorithms in their ability to automatically detect aircraft and their attributes in satellite imagery. CosmiQ curated a dataset of ~600 WorldView-3 satellite images spanning over 200 locations in 31 countries. It includes ~30,000 manually annotated aircraft and 9 fine grain attributes including: aircraft length, wingspan, wing-shape, wing-position, propulsion, number of engines, number of vertical-stabilizers, if it has canards, and aircraft role. The accompanying synthetic dataset is generated via IQT portfolio company AI.Reverie’s simulation platform and features over 46,000 simulated satellite images with ~300,000 airplane annotations.

An example of some of our synthetic data we’ll be working with. The dataset features 6 weather conditions including snow, rain, overcast, dust storm, clear skies, and fog/haze.

The experiments in RarePlanes will address three key areas:

  1. The performance tradeoffs of computer vision algorithms for detection and classification of aircraft role (e.g. light civil) /specific model (e.g. Cessna 172) using blends of synthetic and real training data.
  2. The performance tradeoffs of computer vision algorithms for identification of rare aircraft that are infrequently observed in satellite imagery using blends of synthetic and real training data.
  3. The value of weakly supervised annotations for detection of unique aircraft attributes.

These experiments should help to inform the best practices for using synthetic data in an overhead perspective, and be transferrable to other challenging detection tasks beyond aircraft.

Example of the real dataset (Maxar WorldView 3 Imagery) over the Philadelphia International Airport.

I should note that just like SpaceNet a portion of this dataset will be open-sourced! Stay tuned for details on this and a blog series going deep on the dataset, the features, our experiments, and conclusions that should hopefully inspire some future research.

The DownLinQ

Welcome to the archived blog of CosmiQ Works, an IQT Lab

The DownLinQ

As of March 2021, CosmiQ Works has been folded into IQT Labs. An archive will remain here to showcase historical work from CosmiQ Works that took place July 2016 — March 2021.

Jake Shermeyer

Written by

Data Scientist at Capella Space. Formerly CosmiQ Works.

The DownLinQ

As of March 2021, CosmiQ Works has been folded into IQT Labs. An archive will remain here to showcase historical work from CosmiQ Works that took place July 2016 — March 2021.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store