Refining GPS for More Accurate Tracking
By Jeff Knight, Michael Mazzoleni, and Conrad Stoll
Here’s a story that happens all too often. A runner is training for a marathon, and her goal is to break 3:30. She knows that in order to do this, she needs to run a sub-8:00 min/mile pace for the entire race, so she’s trained accordingly with an app or GPS device. But on race day, she finds she’s hitting each mile marker 15–25 seconds later than expected. What happened?
Many runners learn the hard way that GPS-based apps and devices do not measure distance perfectly. Official race distances are actually measured using a bicycle mounted tachometer (cyclometer), not GPS, which can often overestimate distance. Overestimation errors that result from unfiltered GPS data can really add up. Over a 26.2-mile race it could be over 10 minutes–the difference between qualifying for the Boston Marathon, or not.
As a running app, we care deeply about accuracy. We spend a lot of time talking to people about MapMyRun and accuracy is one topic that always comes up. Given its importance, we are always looking for ways to improve. Recently, we made some changes to the way we filter GPS data in an effort to increase MapMyRun’s accuracy. This was no small effort. It included analyzing thousands of races tracked in the app, hand measuring routes in multiple cities, and running hundreds of tests.
We have very high confidence that these changes increase accuracy for almost all users of the app. But many users will notice that the distance of their typical routes will be somewhat shorter — and any change like that can be alarming. So we wanted to take an opportunity to explain what changed, why we changed it, and why we’re confident these changes make MapMyRun better than ever.
Background
GPS technology was developed by the United States government to provide geolocation and time information for military, civilian, and commercial users. The technology uses signals from multiple satellites to triangulate a receiver’s position on the earth. Prior to the year 2000, GPS data was degraded for non-military applications due to a policy known as “Selective Availability,” which limited its utility for products like fitness apps. Unlocking GPS in the new millennium has given athletes worldwide a convenient method for tracking runs, walks, bike rides, swims, and other activities. The GPS technology that powers Under Armour’s MapMyRun app is tuned to provide a very high degree of accuracy for tracking workouts, and helps athletes around the world to chase and reach their fitness goals.
Over the years, we’ve learned that GPS is not perfect. There is some inherent noise in GPS signals that come from tall buildings, trees and foliage, and other types of interference. This interference delays or obstructs the signal from reaching a smart phone. Noisy GPS signals are a problem for tracking workouts. When a phone thinks it’s on the wrong street, or the wrong side of a river, that’s usually caused by interference. The more severe that interference gets, the greater the margin of error can be.
GPS signals must be filtered in order to provide accurate estimations of distance and speed. Imagine a person running down a busy city street. Their phone’s GPS receiver is having trouble getting an unobstructed signal, and their position dot on the map keeps moving from one side of the street to the other. If you connect the dots between those points and add the distances together, the resulting distance would be longer than the actual street itself. The end result for the workout would be an overestimation of distance and speed (i.e., faster paces, longer distances). We use filters to improve the experience. Filters select only the most accurate points to be part of the line, so that the computed distance will be as close to the actual street’s distance as possible. You will see an example later in the blog.
Developing a New GPS Filter
We’re introducing new GPS filtering technology in MapMyRun to help athletes avoid major GPS errors and overestimations while tracking workouts. We’ve been testing this through multi-month studies and a range of methodologies. Throughout those test we’ve seen that the improved filtering delivers a very high level of accuracy when compared with measured courses, race courses, and results from other products. Let’s go over some of the tests and what we learned from each one.
The first step was to test the existing GPS filtering system to measure its level of accuracy before we made any improvements. We began by looking at data collected in a highly controlled environment. We looked at GPS signals recorded on routes that we measured with a cyclometer, following the preferred method of USA Track and Field. We literally measure these routes by hand! We then compared the GPS-based distance to the cyclometer-determined distance and found that the existing GPS filtering system overestimated distance by 4–6%.
Next, we tested the existing GPS filtering system in a semi-controlled environment. Using technology that identifies races tracked by MapMyRun (as described in US Patent Number 10331707B2), we identified over 500 USA Track and Field sanctioned races (i.e., the race courses were measured with a cyclometer). Comparing the GPS-based distance from MapMyRun, we found that the existing GPS filtering system overestimated distance by the same 4–6% that we saw in our own controlled testing. The similar results in overestimation error with both the controlled and semi-controlled tests gave us confidence in our baseline and gave us a meaningful target to improve upon.
We took a detailed look at our GPS filters and identified several corrections that could improve accuracy, using that 4–6% overestimation as our baseline. We leveraged numerical simulations to optimize our changes to the GPS filters, which reduced the occurrence of individual errors by 50%. This error reduction was primarily achieved by reducing the overestimation tendencies of the original GPS filters. Overestimation is caused by including erroneous GPS points in a tracked workout. The errors in those points cause the total distance to be higher than the actual distance for the workout. Therefore, when the new GPS filters are applied and fewer errors become part of a workout, the speed and distance for the workout will be lower than it would be with the old GPS filters, or without any filters at all.
We validated the new GPS filters by re-running all of the above tests with the new filters. We found that the new GPS filters significantly out-performed the original filters. The new GPS filters reduced overestimation to 2–3%, down from 4–6% with the original filters. We also compared the results for the same tests to the results obtained by Garmin™ wrist-based GPS run trackers, and found that the new GPS filters produced distances to within 1% of the wrist-based run trackers, compared to within 3–5% for the original GPS filters. The margin of error with the new GPS filters is much better, especially at reducing overestimation errors, which means the new GPS filters will better match industry standards and expectations.
How does this impact the long-time MapMyRun athlete?
The honest truth is many athletes will experience a decrease in workout distance. Most athletes will see a 2–3% decrease, but some may experience larger differences. We looked at a large random sample of workouts tracked by MapMyRun and applied both the old and new GPS filters to that data. We found that, on average, the new GPS filters will report a speed and distance that is 2–3% lower than the values reported with the old GPS filters.
The new GPS filters also make large, random, GPS tracking errors far less likely to happen. This should be a relief for many people in areas prone to poor GPS signal. No one likes to see a GPS tracking line that goes off into a lake, or into another part of a city (see the below figure for an ACTUAL workout, before and after the new GPS filter). Large GPS tracking errors can ruin a workout, and we want to prevent these from happening whenever possible.
We do understand that seeing a decrease in distance may be frustrating. We assure you that our intentions are not nefarious. Our intention is to be as accurate as possible. We believe accuracy is honesty and honesty builds trust. We hope that you will allow us to continue to earn your trust through updates like this. We are confident that updates like this will allow us to better help you reach your goals, and improve foundational elements of MapMyRun to enable the next wave of great features.
