SpaceNet Roads Extraction and Routing Challenge Solutions are Released

In March, we concluded the SpaceNet Road Detection and Routing Challenge hosted by CosmiQ Works, Radiant Solution and NVIDIA. Accurate road networks are an important map feature that is required for everything from logistics planning to turn-by-turn directions. Currently, road networks are traced by hand from overhead imagery or created from ground surveys. The SpaceNet Road Detection and Routing Challenge tasked competitors to develop algorithms to extract road networks from satellite imagery.

For the competition, SpaceNet released a new roads dataset with over 8000 kilometers of roads labeled, and introduced a novel metric for road extraction, Average Path Length Similarity (APLS). APLS is designed to measure the routing capability of a network graph, as opposed to traditional pixel mask based metrics that do not address the underlying road connectivity. During the two-month long competition, we received 342 submissions from 33 competitors.

We are happy to announce the source code for the top five competitors is now available at the SpaceNet Github repository under an Apache 2 License. Each solution comes with a description of the algorithm by the competitor, as well as instructions for how to build the Docker container, perform the model training process, and perform inference on the SpaceNet dataset using the released source code.

SpaceNet Roads Extraction and Routing Challenge Results:

The top three solutions to this challenge were in fact quite close to each other in overall performance. Individual performance on a per city basis varies significantly, however, as can be seen in Table 1.

Table 1: Validation results from The SpaceNet Roads Extraction and Routing Challenge

SpaceNet Roads Extraction and Routing Challenge Code Details:

The top 5 competitors all followed a similar solution path, which is depicted in Figure 1:

A. Apply a trained segmentation model to the satellite test image

B. Retrieve a to create a raw prediction mask

C. Refine the pixel mask using different post processing techniques

D. Skeletonize the pixel mask

E. Turn the skeleton mask into a graph network

F. Perform Post processing on graph network

Figure 1: Road extraction workflow

Competitor “albu” finished in first place with an overall APLS score of 0.6663. His solution used only the pan-sharpened RGB band and rescaled the imagery. He then split the data into 4 folds and used a resnet34 encoder and a unet-like decoder with skip connections. All training was done on 512 x 512 pixel crops from the training data. Inference was done on the full image with a 22 pixel border. He then turned the probability map of road pixels into a binary image and skeletonized the binary image. Next the sknw package was used to create a graph structure. From there, a series of post processing steps straighten lines, filled in unwanted gaps, and removed extraneous lines. Table 1: shows a summary of the top 5 competitors solutions. We encourage interested readers to check out the Github pages for more details about each specific implementation.

Table 2: Description of Top 5 competitor’s solutions. For more details see each competitors write up and code on Github.

What’s Next:

The SpaceNet team is hard at work creating the next dataset and challenge which will focus on building footprint extraction from off-nadir imagery. Stay tuned for more details this summer.