Published in


Building Footprint Extraction and Damage Classification

Authors: Rohit Singh, Sandeep Kumar;
Contributors: Vinay Viswambharan, Divyansh Jha, Shivani Pathak, Daniel Wilson

At the plenary session of this year’s Esri User Conference, USAA demonstrated the use of deep learning capabilities in ArcGIS to perform automated damage assessment of homes after the devastating Woolsey fires. This is a critical task in damage claim processing, and using deep learning can speed up the process and make it more efficient. This blog post shows how this was done, from training the deep learning model to detecting damaged homes using it.

Damage assessment using drone imagery and deep learning

The first thing that deep learning needs is training data to look at and learn from. To build the model, we need to teach it what a damaged home looks like in drone imagery. For this, we used ArcGIS Pro to manually label a few hundred homes, marking them as damaged or undamaged by looking at their drone imagery. A GIS layer of the existing building footprints was used for this. We created a new field in the building footprints layer and gave an attribute of damaged or undamaged.

Exporting training samples

These were used to export training samples in the ‘Labelled Tiles’ format using the Export Training Data for Deep Learning tool in ArcGIS Pro. Pro has recently added support for exporting training data in the Labelled Tiles format to support this task.

Labelled Tiles used for training Damage Classification model

Training the Damage Classifier model

Once we had the training data ready, we used ArcGIS Notebooks and the arcgis.learn module in the Python API to train the model. Essentially, all it took was about 10–12 lines of boilerplate arcgis.learn code that looked like this:

Notebook for training damage classification model

ArcGIS Notebooks is pre-configured with the necessary deep learning libraries, so no extra setup was required. Training the model using a ResNet34 architecture gave over 99% accuracy. This isn’t surprising, as deep learning is especially good at image classification tasks like these.

Building damage classification — red ones are damaged


By the time the notebook completed running, it had already trained the model and updated the layer classifying all available building footprints as damaged or undamaged. A web map with results of the damage classification is shared here.

Resulting layer of damaged buildings overlaid on imagery. See webmap.

It turned out that due to newer developments, the building footprint layer that we had available was outdated — it did not contain all the buildings. Getting high quality geospatial data like this is labor intensive and costly. Could we use deep learning to extract updated building footprints from imagery? It turned out that we could! See how we did that in the next section.

Extracting building footprints

We used the existing building footprints as training data to train another deep learning model for extracting building footprints. This time, the model that we had to train was a pixel classification model, wherein each pixel belonging to a building needed to be labelled as such.

Pixel classification for extracting buildings

Exporting training data

ArcGIS Pro has support for exporting training samples in the Classified Tiles format for training such models. All we had to do was convert the existing building footprint layer to a classified raster and feed it into the tool.

Training the model

Training the model was straightforward. Our training notebook was just like the one used earlier, but this time we used a U-Net model in arcgis.learn for pixel classification.

Training a UnetClassifier for extracting building footprints

The model trained here was used to segment out the pixels belonging to buildings. ArcGIS Pro readily supports models saved by arcgis.learn.

Extracting building footprints

Going from imagery to raster masks and vectorized building footprints

We used Classify pixels using deep learning tool to segment the imagery using the model and post-processed the resulting raster in ArcGIS Pro to extract building footprints. To post process the classified imagery we used Model Builder — it works great for repetitive workflows, especially when we are experimenting with parameters to get the best results.

Post processing using Model Builder

The results of the building footprint model are a treat to see:

Building footprints extracted using automated process — no hand editing!

A web map with the extracted building footprints is shared here. Having such high quality building footprints generated automatically with no human intervention is a testament to the progress AI has made. Doing this end-to-end workflow in ArcGIS made the process easy and straightforward with all the necessary tooling in place.


Finally, for extracting the updated building footprints in our entire area of interest covering over 60,000 buildings, we used ArcGIS Enterprise with distributed Raster Analytics capability and scaled out the processing so it could be done within 20 minutes — that’s a rate of generating 3,000 building footprints every minute!

Time is a critical resource, be it for processing insurance claims, or for providing aid to those affected by natural disasters like hurricanes, floods or fire. Applying deep learning techniques like these to assist with damage assessment after such unfortunate events can indeed be a life saver.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Rohit Singh

Rohit Singh

As the director of Esri’s R&D Center in New Delhi, Rohit Singh (@geonumist) leads the development of data science, deep learning and Geo-AI solutions in ArcGIS.