Collecting Metrics, Going Mobile & More
Insights from a MapRoulette Meetup
MapRoulette has become the go-to micro-tasking tool for OSM, thanks in large part to its diverse user base ranging from local experts to corporate organized mapping teams and everyone in between. Since its inception, MapRoulette has been a powerful tool for improving data quality in OSM — one task at a time. Users are served tasks they can complete in the OSM editor of their choice. To date, over 500,000 tasks have been completed in MapRoulette! Critigen’s Open Data & Development team regularly uses MapRoulette to publish Atlas-Checks results as well as online mapathons in an effort to contribute to OSM.
A colleague and I recently attended a MapRoulette meet-up in Seattle. There were around 25 people who attended from different backgrounds — all representing important stakeholders for MapRoulette. The general theme of the day was slated to be “MapRoulette Metrics” but a wide range of topics were discussed.
Collecting Metrics from MapRoulette
Metrics have been a hot topic within the MapRoulette community. There has been a push, led primarily by corporate mapping teams, to engage in a broader discussion about how metrics can be collected from MapRoulette. The conversation at the meetup revolved around two types of metrics: changeset metrics and task status metrics.
Changeset Metrics
Examples of changeset metrics include total kilometers of road added per task, number of nodes created per task, or time spent editing per task. In these cases the metrics are based on information that can be found in OSM via the changeset ID. The current mechanism MapRoulette uses to link each task to its respective changeset ID is imperfect, and needs to be enhanced for greater accuracy and reliability.
Task Status Metrics
The second type of metrics is based purely within the MapRoulette environment. Each task has a status that can be assigned to one of the following seven values:
- 0 — Created
- 1 — Fixed
- 2 — False Positive
- 3 — Skipped
- 4 — Deleted
- 5 — Already Fixed
- 6 — Too Hard
Below is an example of how these task statuses are surfaced to the user within the MapRoulette UI.
These statuses and any changes to them are recorded in the task history. Task histories can be aggregated and used to gather metrics on challenges or projects. This type of metric can be obtained easily using the MapRoulette API.
Quick Fixes
Beyond our discussion of metrics, another topic was a new functionality in MapRoulette called “quick fixes”. These are tasks that can be completed by the user without ever leaving MapRoulette! This type of task can be an efficient way to clean-up OSM data, but the tasks must be very simple (i.e. a yes or no question). Below is an example of a quick fix challenge where the user is given a set of tasks that may benefit from a “lanes = 1” tag. The user can either accept, reject, or skip each task. If they accept, that tag will be updated directly in OSM.
So far the only testing that has been done has involved simple tag changes. There is some interest in expanding into geometry-based quick fixes, but this is still being tested. Our discussion revolved around the best way to roll out quick fixes and what they would be most useful for.
Mobile
A good amount of time was spent discussing the potential for a MapRoulette mobile app. One cool idea was to allow users to create tasks in the field, that armchair mappers could work on in the usual desktop format. Another idea was to serve tasks that require field validation in quick-fix format so a mobile user could complete them within the app. There was an emphasis on determining the value that the app would provide and how that would compliment the current MapRoulette workflow. In any case, app development is a large project and the consensus was to start small.
User Groups
MapRoulette has functionality to track project progress on the task level. However, it’s not easy to understand the diversity of users that are accessing your project. For example, is there one super-user completing all the tasks or multiple users? Introducing user groups may help in collecting user metrics. In order to harvest any user related information, user groups would have to agree on some tracking or monitoring mechanisms. Corporate use-cases include managing the editing/review process with editors. A more community-aligned use-case would be allowing users to join a user group based on their interests (i.e. address mappers or reviewers). MapRoulette could suggest challenges that are relevant to the user’s interests if this information was available.
Community Engagement
Martijn van Exel, MapRoulette’s creator, was present at the meeting. He continues to be integral in guiding MapRoulette’s development as an important open source mapping tool. An important piece to this is the continued engagement of the OSM community with the main priorities of retaining avid MR users, and attracting new mappers. There was a spirited discussion about the potential for further gamification of MapRoulette and how to do more outreach through Twitter, Slack, and mapathons.
Summary
Overall, this was an inspiring meeting to attend and witness so much personal and organizational investment in MapRoulette. It’s clear that it remains an important asset to the larger OSM community. There’s a lot of excitement around its continued development, but the project is in need of engineers and developers to engage and begin contributing. This will be key, as any development of the project is limited by personnel and time constraints. Critigen continues to contribute to MapRoulette by regularly pushing challenges, and we recently started working on a Python client for the MapRoulette API that we’re super excited to roll out in the near future.
Want to contribute to OSM? Feel free to contribute to our GeoWeek Virtual Mapathon Project that we blogged about earlier this year!
