Can Users Estimate Distance on Web Maps?
A study of how people perceive distance.
This is Part 1 in a series about distance perception on web maps. The data and experiment were part of a month-long summer independent research project I conducted for my studies in the Masters of Liberal Arts and Sciences program at UNC Asheville.
How much crime occurs within a mile of your house? If you live in suburbia like me, you’re probably thinking, “Not much.” But imagine interacting with a website that can provide details about crime occurring within a one-mile radius of a certain location. If the location is your house, you may find that there’s a lot more crime happening than you expected. The mismatch between what you expected and what you discover may cause you to question the validity of the data—and the website.
What causes this mismatch? I’ve seen this scenario play out over and over again on map-based websites that I’ve built, and I’ve always wondered what causes it.
Could it be that we’re just bad at estimating how far one mile is on a map?
Some people recommend that we use time-based measures—such as walk time or drive time—rather than Euclidean measures of distance, claiming that these are more relatable to humans since we do them every day. The claim has no data to back it up, and very little literature that specifically ties walk or drive times to distance estimations.
We also need to consider the potential effect of cognitive biases on variables such as crime. Estimations of crime may be affected by biases, such as the availability heuristic described by Tversky and Kahneman—but there we would expect overestimations, not underestimations. To see if the geographic variable type “crime” is playing a role, we’ll add a few other geographic variables in the form of prompting questions.
This study tries to determine if (i) perception of map distance, and (ii) the question asked (such as crime) contribute to the observed behavior. The intention is to help designers of mapping websites match users’ mental models of distance, thus improving usability.
Having evidence to base our designs on goes a long way toward building better and more usable websites.
The Experiment
To study how people perceive distance on web maps, I built a dynamic mapping website:
The study is still open, so take a break and participate!
The site is a three-step questionnaire that acts similarly to a Google Form. All data is collected into Google Sheets using its API (Application Programming Interface). Participants are acquired through email, email groups, Twitter, LinkedIn, Facebook, Slack, and word of mouth. Participants are also asked to share within their networks.
In the first step, the participant is asked one of three randomly selected questions about searching for a location. The potential questions are:
- Search for a location you care about (“care”),
- Search for a location to find out about crime (“crime”), or
- Search for a location to find out about the closest pizza place (“pizza”).
The “care” and “pizza” questions are included as foils to determine if cognitive bias may affect the estimation of the “crime” variable—we want to know if the question asked might affect the judgment of distances. Including another geographic variable such as “pizza” and a potentially more neutral question such as “care” may help isolate the effect the question has on distance estimations.
In step 2, the participant is asked one of three randomly selected distance questions:
- Draw a circle that represents 1 mile (“mile”),
- Draw a circle that represents a 5-minute walk (“walk”), or
- Draw a circle that represents a 5-minute drive (“drive”).
The step 2 question is prompted with the directions:
Click on the “Draw a circle” button. Then click on the map to establish the center of the circle. Move your mouse over the map, when the radius of the circle best represents the distance, click on the map again.
In step 3, the participant is asked:
Submit the current circle as your answer or redraw the circle by clicking on the “Draw a circle” button.
Once a submission is made, the user can no longer participate. Clearing the browser cache does not remove the block. However, a knowledgeable participant could get around the block by using another device or browser. There is no way of knowing if a participant did so; the assumption is that they did not.
To check for accuracy of submissions, I’ll use one-sided t-tests for each category of distance (mile, walk, drive), which will inform us if the mean of the logged submitted distances is different than the expected distances. For the one-mile category, I expect the participant to estimate 5,280 feet; for the walk category, I expect the participant to estimate 1,320 feet; and for the drive category, I expect the participant to estimate 18,480 feet. The results of the t-tests must have p-values > .05 to be considered “accurate” for the expected distances of each category.
I’ll repeat the measure of accuracy using a one-sided t-test for the question asked (care, crime, or pizza). To evaluate if the questions had any effect on participants’ estimates, I’ll also use one-way ANOVA tests. If there are any differences between the questions, I’ll run post-hoc pairwise tests to identify the specific question that may have affected the estimated distance.
Results
As of this writing, 277 participants agreed to participate in the study. Of those, 42 did not submit an answer. In usability tests, some users—especially on mobile devices—could not correctly draw the circle (they tapped the screen twice, which resulted in a circle under 10 feet). By the third round of usability testing, this did not happen—but it could still happen, and did. Because of this error, I removed three circles measuring less than 10 feet. Four circles greater than 60,000 feet (more than three times any of the expected distances) were also submitted and removed, leaving 216 observations. I refer to the dataset consisting of 216 observations as the full dataset.
As we can see in Figure 1, the dataset is not normally distributed. This will violate the assumptions required by the t-test and the ANOVA test.
Logging the submitted distance, as observed in Figure 2, satisfies concerns with a normal distribution.
In the following map figures, the submitted circles are superimposed over the City of Asheville, North Carolina. By moving the center point of the circles to Asheville, we can ensure the anonymity of participants and facilitate the comparison of circle distances on a map.
Examining Figures 2–4, we see that participants underestimated the distance for the “mile” category, overestimated the distance for the “walk” category, and considerably underestimated the distance for the “drive” category.
Table 1 quantifies the accuracy of estimations for each distance category. Any distance category where the p-value > .05 is highlighted in green. A p-value > .05 indicates that the mean of the submitted distances is likely not different than the distance we expected.
Only the “walk” distance category has a p-value greater than .05, thus we can consider participants accurate at estimating a 5-minute walk.
Next, I examined whether the three questions asked of participants (care, crime, and pizza) influenced the estimation of distance, considering the difference between the questions and the accuracy of the estimations.
Table 2 quantifies the accuracy of estimations for each distance category and the questions asked. Any distance category where the p-value > .05 is highlighted in green. A p-value > .05 indicates the mean of the submitted distances is likely not different than the distance expected.
For the “mile” distance category, participants are accurate when asked the “care” question. For the “walk” distance category, participants are accurate when asked the “care” and “pizza” questions. For all other distance categories and questions, participants are not accurate at estimating distances.
The figures below display the p-values and confidence intervals and the maps for each distance category.
So…are participants’ estimates different depending on which question was asked? To determine the answer, I looked at the results of ANOVA testing, as reported in Table 3. Any distance category where the p-value > .05 is highlighted in green. A p-value > .05 indicates that there is likely no difference between the estimated difference for each distance category and question asked.
For all distance categories, the questions asked do not affect the estimations of distance.
Conclusions
Directly addressing the hypothesis that users are not able to estimate one mile on a map is likely true. The mean of the submitted distances and the expected distance are significantly different. Furthermore, the difference of one mile tends to be underestimated by an average of 1,486 feet.
This underestimation supports the hypothesis that people cannot estimate one mile on a map.
This also helps explain the mismatch between the amount of crime a user expects to find and what they actually discover.
To put this in perspective, the mean submitted circle has a radius of 4,578 feet and an area of 65,841,762 square feet. The expected circle has a radius of 5,280 feet and an area of 87,582,576 square feet. The difference between the two is 21,740,814 square feet—which could result in as much as a 25% difference in the amount of crime in the submitted one-mile area compared to the expected area. Of course, this assumes the geographic variable (crime) is distributed normally across the surface, and that the results of this study are reproducible and accurate.
We also find that a 5-minute walk time is an excellent measure to help users make accurate estimations of distance. It would be interesting to test other walk-time intervals to see if people continue to estimate distance accurately using time measures of distance.
Advice that walk time is something that users can better relate to, or will help users in estimating distance, is valid.
The “drive” category is quite another issue. The tendency to underestimate drive time is possibly due to cognitive bias from negative emotions experienced in traffic. Some participants provided feedback that traffic flow was not accounted for in the expected distance of 18,480 feet.
In this study, we are assuming “as-the-crow-flies” distances. Roads, of course, do not follow this measure. It is possible that we need to lower the expected distance for drive time. If we reduce by 25%, the expected distance would be 13,860 feet. Running a one-sided t-test with the reduced distance results in a p-value of 4.02E-07—less than .05. However, we still cannot say that they are the same.
It’s probably a bad idea to use drive time when wanting a user to make an accurate judgment of distance.
For the most part, we find that the questions asked—crime, care, and pizza—do not affect the accuracy of distance estimations, except for the “care” question with the “mile” and “walk” distance categories. There is likely no difference in the estimations between the questions for any of the distance categories.
The questions asked are most likely not influencing distance estimations.
A few other factors may influence a participant’s ability to judge distance—factors such as screen size, size of the browser window, mobile vs. desktop, and zoom level. This information was captured along with the results reported here. Zoom levels and elapsed time are two factors that warrant further study.
This study details a few ways to better match the mental models of distance held by our users—which, generally speaking, provides a better user experience.
I continue to collect participants in this study. In Parts 2 and 3 of this series, I’ll look into how distance estimations are affected by the amount of time it takes a user to draw the circle and the map zoom level when the circle was drawn.
Many thanks to Dr. Mike Neelon for his guidance and support on this project!
