Introducing Mashnet

SharedStreets has released a new programming library for merging overlapping street network datasets, called Mashnet. Mashnet can be used to conflate two or more street networks. It reads OpenStreetMap pbf road graphs and GeoJSON LineString features by default, and can be adapted to read other graph representations. It is open source under an MIT license.

Mashnet represents street networks as a graph, with streets and intersections modeled as graph edges and nodes. This allows Mashnet to cleanly merge conflicting geometry from various road networks while preserving topology used for routing and cartography.

Mashnet represents the street network as a graph data structure, with streets stored as graph edges and intersections represented as graph nodes. When loading a new network, Mashnet performs an algorithm called “graph normalization”, which transforms the network into a series of edges and nodes that is consistent across sources. New streets are then proposed and Mashnet is able to incrementally normalize the graph. This allows new roads to merge into the network without breaking important connections needed for cartography and routing.

Mashnet uses a custom graph data structure for fast performance. R-Trees are used for identifying match candidates quickly.

Mashnet is designed for performance, but also for partition friendly architecture. This allows conflation operations to be scaled across many computers to enable global scale networks to be merged. To achieve this, Mashnet stores geometric data in a series of R-Tree structures that allow for efficient spatial querying, cutting down on unnecessary operations.

In this gif, the new street in pink is merged into the existing network. Mashnet is able to detect newly created intersections and use the existing nodes for clean topology.

When a new street is proposed, Mashnet first decides if the street crosses any existing intersections, and will split the proposed street into multiple chunks if needed. Next, the proposed streets are run through a neural network classifier that is pre-trained to identify whether the new street is a match for any existing streets. If it is, Mashnet can then merge the associated metadata into the existing record. This allows a road network with good metadata, such as speed limits or bike lane accessibility, to be merged into another network with good geometry, creating a new map that is better than any single source. The neural network comes with a pre-trained model based on millions of synthetic edits, generated by perturbing the street network of Hawaii in OpenStreetMap.

The artificial neural network shipped with Mashnet is trained on millions of roads that have been perturbed to simulate differences between basemaps. The model is shipped pre-trained for ease of use.

If no existing match is found, a new street has been identified. Mashnet is then able to merge the new geometry into the existing graph without disturbing the existing geometry. This is helpful for identifying new roads which can be committed directly to the database or logged to a task manager for manual review.

When a street is proposed, Mashnet will classify some edges as new geometries. In these examples, Mashnet has identified numerous streets in recently developed subdivisions where the local DOT had fresh data, which the OpenStreetMap community had not yet mapped.

Mashnet is a new approach to street network conflation that aims to balance performance and robust merge quality, without requiring the user to manually account for every subtle edge case. We are currently testing Mashnet to merge datasets like OpenStreetMap to other road networks such as ML derived aerial networks as well as traditional road networks maintained by DOTs.

If you would like to try out Mashnet, the code is open source, and we can help you get started with questions on Github.