The SpaceNet 8 Flood Detection Challenge: Announcing the Winners
Editor’s note: SpaceNet is an initiative dedicated to accelerating open-source, artificial intelligence applied research for geospatial applications, specifically foundational mapping (i.e., building footprint and road network detection). SpaceNet is run by co-founder Maxar and our partners Amazon Web Services (AWS), IEEE-GRSS, Oak Ridge National Laboratory and Topcoder.
SpaceNet 8 has officially concluded and we are excited to announce the top 5 winners. Below, we’ve provided a summary of the overall challenge goals and listed the winners. Further details about the winning solutions and analysis will be shared in a subsequent blog post.
The Challenge
The SpaceNet 8 Flood Detection Challenge focused on infrastructure and flood mapping related to hurricanes and heavy rain that cause route obstruction and significant damage.
The goal of SpaceNet 8 was to leverage existing datasets and algorithms from SpaceNet Challenges 1–7, as well as new training data and a baseline algorithm, then apply those to a real-world disaster response scenario. This challenge also expanded the task to a multiclass feature extraction and characterization problem.
The SpaceNet team’s paper, “SpaceNet 8 — The Detection of Flooded Roads and Buildings,” was presented in June 2022 at EarthVision, a workshop held as part of the Computer Vision and Pattern Recognition Conference. The paper describes both the dataset and baseline algorithm in detail. The SpaceNet 8 dataset will remain available for download via the AWS Open Data program. Directions for accessing the dataset can be found in The SpaceNet 8 Flood Detection Challenge: Dataset and Algorithmic Baseline Release blog post.
The Winners
The overall winner for SpaceNet 8 is team KARI-AI (username Ohhan777) from South Korea with a score of 66.998/100. Our partner Topcoder asked the top 10 teams from the public leaderboard to submit their algorithms, as Docker containers for blind inferencing tests against the challenge’s “mystery city.” Using blind test data allowed us to evaluate the performance of proposed solutions against similar data as the public datasets we provided while ensuring the solutions could be generalized. The full results of final testing are summarized below.
Interestingly, all winners used available data sources from previous SpaceNet challenges (e.g., SpaceNet 3 and 5 for roads and SpaceNet 1 and 2 for buildings) as well as strong data augmentation during training to enhance their model’s generalization capabilities. This approach was particularly important in SpaceNet 8 because the final evaluation was performed on an area that is spatially distinct from the training areas.
Congratulations to all SpaceNet 8 winners! We look forward to sharing further details on each approach taken for the winning solutions in our next blog post.
