This tutorial was originally part of a joint workshop between DirectRelief, NetHope, and Facebook at the 2019 NetHope Global Summit in San Juan, PR.
The team I work with at Facebook has been busy releasing and updating RapiD, a version of the primary OpenStreetMap (OSM) iD editor that helps every mapper make fast, high-quality, and accurate edits using roads suggested by our AI-assisted road import process. We’ve been expanding OSM in countries like Thailand, Indonesia, and now Malaysia with a robot-assisted process that pairs advanced imagery processing to find likely roads in fresh aerial imagery with human oversight to confirm and add roads to OSM. Initially these tools were available internally but now they’re easy to use for anyone. In combination with up-to-the-minute OSM extracts it’s possible to integrate OSM into a GIS environment in a single session using just browser-based tools.
In this tutorial I’ll show how to view robot-suggested additions to OSM with RapiD, save them to the map, get your data back out with Protomaps Extracts, and use the resulting data in QGIS. I’ll also show how RapiD can be used with popular OSM tools like Maproulette and HOT Tasking Manager, for when you want to help build the map but you’re not sure what to do.
Before You Start
Download and install a recent version of QGIS, the free and open source Geographic Information System from qgis.org. Version 3.4 LTS (long-term support) lets you open OSM PBF files directly without needing to first convert them to Shapefile or GeoJSON.
Add AI-Assisted Roads
When you zoom in far enough, you can see features to edit. Here there are some orchard boundaries marked out with a few tracks already mapped. RapiD also shows roads colored purple. Purple roads are AI-assisted features where RapiD thinks that no OSM road exists but there might be one in the aerial imagery.
These purple roads are not in OSM, and you can add them to the map individually as you verify that they match the underlying imagery. When you click on a road, RapiD adds it to the map and it changes from purple to another color depending on its type.
Click around to add a few more roads in this area. The existing roads on the orchard were tagged as “unmaintained tracks”, so we’ll mark the new ones with the same tag to keep this map consistent with existing local mapping efforts.
As you add roads, a number of features added or changed shows in the upper-right corner. Here we’ve added over 20 new segments of connected track in this orchard. Now we can hit the “Save” button to upload them to OSM.
The first time you ask to save, RapiD will prompt you with an OSM authorization window. Click “Grant Access” to allow RapiD to upload these edits using the correct account.
Add some comments about the edit, hit “Upload”, and you’re done. RapiD might offer a few warnings about disconnected roads that you’ll want to fix before saving your changes.
Extract OSM Data to QGIS
Let’s get our newly-mapped data into QGIS with Protomaps Extracts at protomaps.com/extracts. This service reads new edits from OSM once per minute. Watch the “last update” time on the left side of the window to be sure your recent edits are included. You may need to wait 2–3 minutes to be sure.
We’ve named our extract “Bintulu Tracks” and made a rectangular selection around the area where we updated the orchard. Click “Create OSM PBF” to ask for an extracted piece of current OSM data for this area with recent changes.
Protomaps Extracts shows a set of progress bars as it builds the extract. A small and sparse area like this should just take a few seconds, while a larger or denser area might take longer to prepare.
Finally click the “Download PBF” button to get a file called Bintulu Tracks.osm.pbf.
Now we can switch to QGIS and open a new layer. OSM PBF files are supported directly in newer versions of QGIS and they can be opened in the same way as Shapefiles and other vector layers. QGIS creates five feature layers from the free-form tagged data with different geometry types and field definitions. Roads end up in the “lines” layer, buildings in “multipolygons”, and points of interest go in “points”.
Now we can see and use the new roads we just created alongside other GIS data, export them to other GIS formats, and use them in further workflows.
Don’t Know Where To Edit?
What if you don’t know where to spend your effort in OSM? Several community tools have you covered. RapiD is already integrated into Maproulette, a spin-the-wheel style editor that directs your attention to a stream of easy, small changes and fixes. To begin, sign in with an OpenStreetMap account at maproulette.org.
Maproulette includes location challenges, and Facebook has created one for the East Malaysia region. Click “Start” to work on this challenge.
Maproulette will choose an area to map. RapiD may not be the default selected editor, so you may need to select it from the drop-menu above the “Edit” button.
When you click “Edit”, RapiD will open in a new tab and work normally. Edit roads and other features in the selected area, save your changes, and close out RapiD as above to return to Maproulette. Finish up by telling Maproulette what you’ve done so the overall challenge can be completed.
Humanitarian OpenStreetMap Team (HOT) also presents a selection of mapping tasks. HOT partners with organization such as the Red Cross to focus attention on areas in need of urgent attention due to ongoing or predicted natural disasters. HOT has created a special version of the Tasking Manager that integrates RapiD at tasks-assisted.hotosm.org, and you can log in with your OSM account just like with Maproulette.
The Tasking Manager flow is similar to Maproulette’s. There are instructions to select a challenge and a task, and the option of selecting RapiD for editing with machine-assisted roads.
Tasking Manager is especially useful for directed group efforts. For example, a large group of mappers might use it to break up an area into small pieces and edit in parallel. Everyone’s changes will be saved to OSM as they work. At any point in the exercise or upon completion, the saved OSM data can be extracted via Protomaps Extracts and used in a GIS environment.
This tutorial has shown how to incorporate suggested edits to OpenStreetMap provided by RapiD, Facebook’s AI-assisted OSM editor. New data is immediately available in Protomaps Extracts service for inclusion in GIS workflows. RapiD is also available via popular OSM tools like Maproulette and HOT Tasking Manager.
Thanks to encouragement, feedback, and testing from Drishtie, Jeff, Brandon, Andi, Alex, Mercedes, Andrew, Jen, Eugenia, Kevin, Danil, Stephanie, Tom, Melissa, David, Greg, and Rolando.