SpaceNet 6: A First Look at Model Performance

Preface: SpaceNet LLC is a nonprofit organization dedicated to accelerating open source, artificial intelligence applied research for geospatial applications, specifically foundational mapping (i.e., building footprint & road network detection). SpaceNet is run in collaboration by co-founder and managing partner, CosmiQ Works, co-founder and co-chair, Maxar Technologies, and our partners including Intel AI, Amazon Web Services (AWS), Capella Space, Topcoder, IEEE GRSS, the National Geospatial-Intelligence Agency and Planet.

Introduction

In this blog we will begin to explore how some of the complexities of Synthetic Aperture Radar (SAR) affected the performance of the winning SpaceNet 6 algorithms. SAR sensors are unique as data is generated by having the sensor actively illuminate the ground rather than utilizing the light from the sun as with optical images. This means that when we look at a SAR image the brightness of each pixel depends on the amount of energy the SAR sensor transmits and receives back at the sensor (known as backscatter). The amount of backscatter received is dictated by the material properties, physical shapes, and the angle from which objects on the ground are viewed. This also means that SAR sensors cannot detect color, but rather the types of backscatter the SAR sensor is receiving. The SAR data in SpaceNet 6 is also captured from an average off-nadir perspective of ~35°. Such off-nadir perspectives, combined with the active sensing of SAR, leads to two further challenges: SAR layover and building occlusion. If you’re interested in understanding more about how SAR works we recommend you read out primer blogs (SAR 101 and SAR 201) on this topic.

In summary, this blog will examine:

• How does building height and size affect model performance?
• How do models trained on optical data compare to the ones trained on SAR in Rotterdam?
• Drawing from the lessons learned in SpaceNet 4: What is the affect of look angle on model performance?

Building Height and Size

Our first look at some of predictions derived from the winning SpaceNet 6 algorithms explore two aspects of the building footprint dataset: building height and size. We extract…