Image for post
Image for post

SpaceNet Road Detection and Routing Challenge — Part I

Adam Van Etten
Oct 3, 2017 · 7 min read
Image for post
Image for post
Figure 1. OSM data screenshot.
Image for post
Image for post
Figure 2. Two regions of Khartoum demonstrating some of the issues of OSM labels when overlaid on SpaceNet data. Left: The east-west road through the middle of the image is not captured by the orange OSM road labels. Right: Poor registration often leads to roads that pass through buildings (yellow polygons denote SpaceNet building labels).
Image for post
Image for post
Figure 3. Example of the pitfalls of the pixel-based F1 metric. Left: Ground truth road mask in green. Middle: proposal mask in orange, which achieves a decent F1 score of 0.82 since most pixels are correctly labeled despite some inconsistencies in road width. Right: proposal mask in cyan, which achieves a superior F1 score of 0.95 since fewer pixels are classified incorrectly. For routing purposes, however, the rightmost plot is clearly inferior since it misses an important intersection and severs a road. The fact that pixel-based F1 scores incentivize the rightmost plot over the middle plot is suboptimal.
Image for post
Image for post
Figure 4. Demonstration of path length difference between sample ground truth and proposal graphs. Upper Left: Ground truth graph. Upper Right: Proposal graph with 30 edges removed. Lower Left: Shortest path between source (green) and target (red) node in the ground truth graph is shown in yellow, with a path length of ~948 meters. Lower Right: Shortest path between source and target node in the proposal graph, with a path length of ~1027 meters; this difference in length forms the basis for our graph similarity metric. Plotting is accomplished via the excellent osmnx package.
Image for post
Image for post
Equation 1. APLS metric. Node a’ is the node in the proposal graph G’ nearest the location of node a in the ground truth graph G. L(a,b) denotes a path distance in the ground truth graph G, and L(a’, b’) denotes the path length between the corresponding nodes in the proposal graph.
Image for post
Image for post
Figure 5. Left: Path length differences for all 1720 possible routes. Right: Path length difference histogram. Most routes are the same between the proposal and ground truth graphs, resulting in the many blue dots on the left plot and the large spike at path length 0.0 on the right plot. There are a few missing routes due to the disconnected node in the center of the proposal graph, and this results in the red dots on the left plot and the small spike at path length 1.0 on the right plot. Applying Equation 1 to this data yields APLS = 0.81.
Image for post
Image for post
Figure 6. Comparison of F1 and APLS scores for a given proposal. Left: Sample satellite image with ground truth mask overlaid in orange. Middle: Proposal mask in yellow, yielding an F1 score of 0.72. Right: Proposal graph (red) inferred from the proposal mask; missing intersections and road segments are heavily penalized, yielding an APLS score of 0.25.

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab…

Adam Van Etten

Written by

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab dedicated to exploring the rapid advances delivered by artificial intelligence and geospatial startups, industry, academia, and the open source community

Adam Van Etten

Written by

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab dedicated to exploring the rapid advances delivered by artificial intelligence and geospatial startups, industry, academia, and the open source community

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store