By Sarah Popelka
This episode is a synthesis of the 3rd episode of our series The Future of Urban AI, co-produced with Cornell Tech.
When people think about urban assets and infrastructure, certain elements particularly stand out in their minds. Utility networks supply the intake and expulsion mechanisms that are necessary for urban metabolism and functioning. Roads and other transportation systems convey people and goods throughout our cities and beyond. Streetlights illuminate our public spaces. ICT infrastructure keeps us connected. But what about trees?
It’s easy to overlook the leafy branches that look over us. And yet, tree canopies provide numerous positive effects on urban life. According to Sara Beery, an ecology/conservation-focused computer science researcher and incoming professor at MIT, these include promoting urban biodiversity, reducing air pollution, providing carbon sequestration, reducing energy use by providing shade/ natural cooling, mitigating extreme heat islands, and promoting physical and mental health. In light of these benefits, many cities invest considerable amounts of money in purchasing, planting, and maintaining the health of their trees. However, access to urban greenspace is not universal, and Beery pointed to studies exploring both socioeconomic and racial bias in urban forest distribution across cities. Advocacy groups, such as Tree Equity Score, use automated tree canopy maps generated from aerial remote sensing data to identify neighborhoods that are underserved in terms of the benefits provided by urban forests and advocate for more inclusive access to trees. Tree canopy coverage only goes so far: to optimize tree planting to reduce inequity, mitigate urban heat islands, and build resilient natural infrastructure that can adapt to future climate change cities need to know the location and species of individual trees. Within the realm of urban forestry planning alone, municipal governments will sometimes spend millions of dollars on tree inventories and censuses. These data collection efforts provide critical information on the existing state of urban forests and guide the planning for their future. However, the cost of traditional tree inventories prohibits many cities from being able to conduct them. The cities that can afford to conduct them usually confine their surveys to a limited geographic area and sample with low frequency. Additionally, the lengthy time frames associated with the current survey methods lead to results that quickly fall out of date.
Beery, in collaboration with Google and other researchers, is utilizing artificial intelligence to investigate automating urban canopy extraction and tree species identification as a lower-cost and more comprehensive alternative to the manual counting methods. She and her team trained deep learning models to identify and classify trees using a combination of aerial imagery and Google Street View imagery (with non-tree, non-sky, non-road pixels blurred for privacy). They tested the scalability of different approaches and identified some important trade offs related to the input data source and the geographic specificity of the training dataset. In terms of input data, they found that aerial data was much more widely available and often more current than Google Street View data, but that Google Street View provided important supplemental visual information that was crucial for species differentiation: models trained on Street View data were 2–3x more accurate at identifying trees than models trained on aerial data alone. Making use of both input data sources proved beneficial. In terms of the geographic specificity of training data, they found that models trained on cities with a broader distribution of different tree species generalized to other locations better than models trained on cities with a higher concentration of a specific type of tree, even in cases where cities had less training data overall. However, the models trained on a combined training dataset (i.e. not trained on a specific city or region) performed better than the more context-based models in most cities.
Recent strides in this work have culminated in Beery and her team releasing the Auto Arborist Dataset, an impressive output that provides 2.6 million tree records accounting for 344 genera across 23 North American cities. She and her collaborators are continuing to work on improving and expanding the dataset by striving to better understand, quantify, and limit uncertainty within their models. Some techniques they are exploring include incorporating Lidar data, accounting for geographic and temporal context (as the same species of tree can have a very different appearance depending on the season and the characteristics of its localized environment), incorporating self-supervision into the models, accounting for “multilabel loss” (i.e. identifying and labeling background trees that appear in the training images, so as not to confuse the model with unclassified data), and bolstering citizen science participation to provide more ground-truth data. Beery’s efforts seek to contribute to the future of urban AI by expanding the geospatial scale of accessible, inexpensive access to urban forest information via automated tree species identification and allowing for improved results across a broader and more diverse geographic scope. Once cities can obtain detailed data for all the trees within their boundaries, without having to conduct costly surveys or censuses, they can better focus their resources on increasing equity and ensuring the resilience of their urban forests well into the future.
By Sarah Popelka, Research Assistant at Urban AI
