Can we predict which Renew Atlanta projects have been recommended for funding in the rebaselining?
Due to funding shortfalls, only a fraction of the projects proposed for Renew Atlanta 2015 T-SPLOST funding will be completed under that funding designation. Preliminary funding recommendations have been made. Here, I explore what factors drive the preliminary recommendations and how those relate to the principles underlying the Atlanta Transportation Plan: safety, equity, and mobility.
Data extraction
For predictors to drive the model, I pulled the project scores from the rebaselining document. All scores from project areas 1–4 (Complete Streets, Neighborhood Greenways, Resurfacing, and Traffic Communication Corridors) were imported and compiled. The TCC group has an additional field (“need for replacement”) which was not tabulated because it was missing for the other groups.
Most of the data could be pulled using Tabula; the resurfacing pages were formatted differently and had to be imported manually. These values were checked against the original for errors. The Cascade Road complete street project phases I and II were on a single line, but had different funding decisions (fully funded/design only). This was handled by duplicating, but decrementing the projected readiness for Phase II by 2 points (arbitrary) because it is reliant on Phase I completion.
There are 98 projects and 6 predictors (“ATP Principles” are Safety, Equity, Mobility; “Other factors” are Readiness, Cost, Partnership Funding).
Analysis
1) How well can I predict whether a project is funded or not based on the ATP principles (Safety, Equity, Mobility) and Other factors (Readiness, Cost, Partnership Funding)?
To answer this question, I fit classifier models that took as inputs either (1) the ATP principles or (2) the Other factors as inputs. The model is trained to predict whether the project was funded (“Fully funded”) or not (all other designations). (Ten-fold cross-validation was used, and accuracy is reported on the held-out sets.) For the ATP principles, I fit a linear classifier, which takes a weighted sum of the factors and compares it to a threshold to predict if that project would be funded (Fully funded) or not (anything else). The accuracy of the classifier 58%, which is marginally better than the “no information” level prediction of 56% accuracy. This analysis implies that safety, equity, and mobility did not impact funding recommendations significantly.
Next, I fit a classifier model to the “Other factors.” A linear classifier based on Other factors performed with 84% accuracy. Because these are categorical values (1, 2, 3, 4, or 5) I also fit a decision tree, which had equivalent accuracy. To make sure that this high performance is not simply because a decision tree is a better classifier, I also fit a decision tree using the ATP principles. This performed with 53% accuracy.
To summarize:
When does the classifier make errors?
Because of the small dataset, it is challenging to say much about what errors reveal about decision making. Potentially, these could point to cases in which safety, equity, and mobility did impact the decision. The linear classifier and the decision tree mostly made different errors. There are two projects that were misclassified the same way by both the decision tree and the linear classifier: Lakewood Way (Resurfacing, S/E/M: 0.57, 0.5, 0), and Old Chattahoochee Ave (Resurfacing, S/E/M: 0.17, 0, 0). Lakewood Way’s high safety and equity impact may have been a factor. Old Chattahoochee Avenue does not have a high safety, equity, or mobility score, and it does not have high readiness or partnership funding; referring to documentation on specific projects, it is a 0.1-mile resurfacing project, explaining its lone positive factor: a VERY LOW project cost.
2) How do the factors that predict funding recommendations relate to equity scores?
To answer this question, I analyzed “low equity” and “higher equity” projects. The equity score is the fraction of the project passing through regions identified as ETAs, or Equitable Target Areas. First, there are very few projects with high equity scores: the median equity score across all projects is zero (0). Of 99 projects, 57 have an equity score of zero. Because this is already a small dataset, I divided projects into two groups: equity score of zero (“low-equity”) and equity score higher than zero (“higher-equity”).
First, project cost. Projects with higher equity scores tended to have higher costs. This histogram shows the project count by cost category (Very low/low cost and Medium to Very High cost). Over half of the higher-equity projects are in the high-cost category, while the reverse is true for low-equity projects.
Second, project readiness. Higher-equity projects tended to have lower readiness. Of the high/very high readiness projects, over two-thirds were low-equity projects, and only one sixth (16%) of higher-equity projects were rated high/very high for readiness. However, about 35% of the higher-equity projects fell into the medium readiness category.
Finally, partnership funding. Very few higher-equity projects (21%) have partnership funding, while nearly half (42%) of low-equity projects have partnership funding.
Discussion and Conclusion
I first analyzed whether I could predict funding recommendations for a project listed on the Renew Atlanta re-baselining prioritization list. I can predict with much higher accuracy whether a project is funded knowing the project readiness, matching funding levels, and project cost than using the safety, equity, and mobility scores. In other words, except in edge cases, safety, equity and mobility scores have little impact on whether projects are funded or not.
This is based on a small dataset from the rebaselining prioritization document data. This analysis is based on pooling across 98 projects that were evaluated on the basis of safety, equity, mobility, project cost, project readiness, and partnership funding. This did not include information on absolute levels of funding (in $) or, with the exception of TCC projects, a measure of “need for replacement.” Because of the small size of the dataset, I did not subdivide by whether a project was a resurfacing, complete streets, neighborhood greenway, or TCC project. While the prioritization document recommends against comparing across categories (Complete streets, resurfacing, greenways, and TCC), it is interesting to note that the classifier was highly accurate even when ignoring those categories.
The actual rules used by the classifier could be idiosyncratic: they were based on predicting data, not on logic. For instance, in one trained decision tree, middle-readiness projects were funded if they were high cost, but not if they were low-cost. Moreover, the linear classifer and the decision tree, though they had similar accuracy, learned slightly different rules and made (mostly) different errors. This analysis was meant to infer what general rules governed recommendations, and those recommendations were not perfect. One should not use a model like these to determine whether to fund projects.
Second, I analyzed the relationship between factors that predict funding recommendations and the equity score. I identified one case, out of 98 projects, in which the equity score probably increased the likelihood of funding: Lakewood Way. However, on the whole, high equity scores are anti-correlated with the factors that lead to funding. Higher-equity projects have lower project readiness, lower partnership funding, and higher costs.
Is it a problem that high-equity-score projects are not being funded? After all, one might argue that equity score only reflects geographic location, and not necessarily the people who are traveling through it. Nonetheless, it reflects the level of investment in neighborhoods, and is a start at quantifying something as complex as equity. Moreover, it is a problem that, of 98 projects, 57 had an equity score of zero, and only 22 had an equity score of > 0.5. These numbers may be inflated by small, low-cost projects in wealthy areas, and further analysis based on data not included in the prioritization analysis packet would be required to assess net funding by equity score. Even so, small projects can have a large impact on neighborhoods, and there is no reason that only wealthy neighborhoods should enjoy that privilege. For the cost of repaving Lenox Road as it passes through Morningside, how many sidewalks could be built in Southwest Atlanta, and why aren’t those projects on the lists?
To summarize, re-baselining decisions can be predicted on the basis of project cost, neighborhood partners, and project readiness. This is not surprising, as these factors were identified as supplementing the decision process. What is surprising is that safety, equity, and mobility are statistically irrelevant to funding decisions, and moreover that the decision-making factors are anticorrelated with equity, a principle that guides the Atlanta Transportation Plan.
As a final note: this model predicted recommendations, not actual funding decidions. Meetings for public commentary on the project list in NW and NE Atlanta continue this week. Announcement can be found here: https://renewatlantabond.com/wp-content/uploads/2019/02/Round-2-Meetings-Flyer.pdf
Data source: Prioritization by project type analysis packet, from https://renewatlantabond.com/prioritization/
