Beware the heatmap: feedback loops in shared mobility
by Joseph Brennan, a cofounder of Zoba. Originally from a small town, Joseph has fallen in love with more than a few cities on two wheels — Bangkok on a motorbike, Beijing on an electric moped, and Boston on a Bluebike. Joseph is a graduate of Harvard College and Peking University.
Zoba provides demand forecasting and optimization tools to shared mobility companies, from micromobility to car shares and beyond.
Zoba is building the future of spatial data science, so it shouldn’t come as a surprise that we love maps. But maps can be misleading. Here, I will explore a common — and, in most cases, costly — feedback loop we see mobility operators fall into when starting their spatial analytics efforts. In doing so, I will build on this article by my teammate Evan (read it first if you haven’t). In the piece, I will share our experience with the feedback loop and present a simulation model we built to demonstrate its impact.
When mobility operators launch in a new market their primary concern is rarely maximizing fleet performance — that comes later. They first aim to get vehicles on the road and drive user adoption. Operations crews and general managers tend to place the vehicles initially, often doing a heroic job of capturing demand using local intuition. With some exceptions, the crews tend to deploy vehicles in fairly consistent pattern over time. ¹ ²
As service stabilizes and rides accumulate, the team (often with the help of HQ) starts to look into their historical ride data. Typically, their initial spatial analysis involves visualizing the historical rides and making a heatmap of ride density. Below is a selection of historical scooter rides across multiple operators in Austin. The data is from Austin’s open data portal and the visualization is made with one of our favorite tools, Uber’s open source kepler.gl. The public data we explore here is from scooters, but the phenomenon holds true for all mobility operator’s deployed assets — from car shares to autonomous service vehicles.
And here is a heatmap of those rides, bucketed by volume into Uber’s H3.
The team then comes to an intuitive, but incorrect, conclusion. They surmise that the high usage areas must be in high demand. They then reallocate more vehicles to those areas in the hopes of increasing performance. By altering their deployment of vehicles to match the historical pattern of their utilization, the operator has now entered a demand-supply feedback loop. I have drawn a flow chart for the cycle below, where X is a given spatial distribution of vehicles in a market.
Why is this a problem? By using a heatmap, the operators are trying to understand demand, but they are actually observing what is in large part a byproduct of where they have historically placed supply. People can ride vehicles only in places where there are already vehicles, so utilization is heavily influenced by supply. To illustrate this point, we developed a simulation model that captures the key feedback loop of “following the heatmap.” It is a simple simulation meant to build intuition around the effect we are exploring. We have publicly hosted the source code for the simulation here if you’d like to explore or use it yourself.
In our model, there are 12 locations that represent areas of a city. All are exactly identical and indistinguishable. We assign each location an average demand value of 100 demanded rides per day. Each ride has an equal 1/12 probability of ending at each location.
At the start of the first simulated day, all locations are assigned 20 vehicles. We simulate the rides that occur on this first day and then allocate 6 more vehicles each day thereafter across locations. So, at the start of the second day, there will be 246 (12 x 20 + 6) vehicles. The number of incremental vehicles each location gets is proportional to the number of total rides at that location across all days. In other words, the simulation places more supply where there has been utilization, just as we see operators do in the real world. This simulate-measure-allocate process repeats for a total of 90 days.
The following figure shows typical results. Here, the x-axis represents the number of simulated days, and the y-axis represents cumulative rides at a location; there are 12 curves, corresponding to 12 locations.
Notice how the curves separate into two groups, despite the fact that the model holds demand constant across locations. The higher-performing locations got lucky and generated extra rides in the first few days get more vehicles, which leads them to generate extra rides and get more vehicles, and so on.³ Quickly, this leads to a stark difference in rides between the “have” and “have-not” stations, even though each station is exactly identical and treated exactly the same by the vehicle allocation process. The only difference is random variation in rides generated!
In fact, the results are even starker if we examine how many vehicles are assigned to each location. Here’s a histogram of vehicle assignments on the last day (remember — equal demanded areas, starting with 20 vehicles each):
Half the locations end up with over 100 vehicles assigned, and half have under 25. Three of the locations end the simulation with 20 vehicles assigned, meaning they never earned an additional vehicle. All that from some random variation in the first days!
This is all occurring despite equal demand across locations. The results are even more surprising when we start to adjust demand values. The next graphic represents a simulation where all locations start with 15 vehicles. Half the locations (in red) are assigned demand of 150 vehicles per day, and half the locations (in blue) have demand of only 50. So the high demand locations (in red) have 3x the demand of the low demand locations.
Incredibly, the areas with 3x higher demand did not necessarily get more utilization — some were outperformed by low demand areas. How could this be? Three of the low demand locations did well early and were allocated extra vehicles, which allowed those locations to continue to see rides and earn additional vehicles. On the other hand, three of the high demand locations, by chance, failed to capture rides early. These locations thus weren’t allocated additional vehicles early on, and for the entire simulation these locations generated as few rides as the low-demand locations.
This model illustrates the danger in letting supply follow utilization: because utilization also follows supply, random initial fluctuations are magnified, creating inescapable feedback loops. The real-world introduces far more variation that leads to starker results than in the model. For example, in a real city, some of the high demand areas might not get any vehicles initially, and some low-medium demand areas get quite a few. The end result? Massive missed opportunities in high demand areas that are perpetually undersupplied.
So what’s an operator to do? The most important thing is to realize that this feedback loop exists and you may already be in it. What we recommend not doing is shifting to other fraught methods that will give you other problematic heatmaps. We see this come in a few forms, but most often in the form of pivoting to over-relying on raw search data (such as app open locations).⁴ The rationale is that users will reliably search for vehicles where and when they want them in a market. While this makes intuitive sense, users do not operate in this way. In the real world, user behavior is influenced by historical supply. For example, demand is often censored from search in areas that have been undersupplied in the past (even high demand ones). Users in adequately or oversupplied areas (or areas with many operators) react differently than to those with unmet demand. Using a heat map of search data, then, creates distinct but often equally misleading results. ⁵ ⁶
Zoba develops models that disambiguate supply constraints from demand, allowing operators to see where people would most want vehicles if they could access them anywhere. Below I use one of these models to show how different utilization and demand can look. On the left, a map of utilization for all scooter operators in Austin (units are rides) where purple is low, red medium, and yellow high. On the right, the demand for scooters in that market (units are rides demanded per hour) with a similar color scale. Drilling down into a few areas we can observe that utilization and demand look very different.
Sometimes the two maps agree, often they do not. Allocating more supply to the high-value areas on the left map kicks off the feedback loop I describe above. By contrast, following the demand outputs on the right enables an operator to actively match supply — either through rebalancing, price incentives, or other levers — to demand. Much of our work at Zoba involves integrating dynamic versions of these models into companies’ operational stack to influence supply levers.
Spatial data is at the heart of shared mobility — and the entirety of the on-demand economy — and heatmaps are a powerful tool for exploring it. But maps can be misleading, and in the case of shared mobility can put an operator in danger if used incorrectly. I hope this piece has illuminated one way that can lead to anemic performance and needlessly slow adoption of a mobility service. Be wary.
Footnotes:
¹ At Zoba, we are continually impressed by the ability of local crews to intuit some of the demand distribution in a market. But it is just not possible to intuit your way to knowing the optimal distribution of, say, over a thousand vehicles no matter how well you know a city. Throw in temporal variance (e.g. weather impacts, weekend-weekday variation) and all bets are off. My hometown in Texas is very small and I know it well. I would place 100 scooters there better than most, but I still would not place them optimally.
² In competitive markets, operators sometimes cluster vehicles near other operators. This exacerbates the issue.
³ This happens in actual markets in part because there is temporal variance in where people use vehicles. This can be because of actual fluctuations in demand (e.g. areas near schools have lower demand out of term) or at random (e.g. differences in how fast residents of two areas adopt a service).
⁴ We also often see teams turn to clustering algorithms to identify hot spots. Clustering, here, is essentially the same as the heatmap approach with the same fundamental flaws.
⁵ This is just one of many issues with search data, though that data does contain some valuable information. For more, see this post.
⁶ This censorship problem is different for mobility fleets with deployed assets than those with more short-term variable supply, like ridehailing companies. Over short time horizons, such fleets have relatively fixed and price inelastic supply. In other words, no amount of demand or willingness to pay can conjure up a scooter closer to you. By contrast, ridehailing supply is more elastic — for some price, you can guarantee a ride in any part of a city. In practice, this means that while people may search from anywhere for an Uber (and then decide if they want it on the basis of price, time to pick up, etc.), they are only searching for JUMP bikes when they have reasonable expectation one will be nearby. We will write more about the differences in the underlying economics of these seemingly similar, on-demand markets in the future.
Zoba is developing the next generation of spatial analytics in Boston. If you are interested in spatial data, urban tech, or mobility, reach out at careers@zoba.com.
