Building Footprint Extraction and Damage Classification
Authors: Rohit Singh, Sandeep Kumar;
Contributors: Vinay Viswambharan, Divyansh Jha, Shivani Pathak, Daniel Wilson
At the plenary session of this year’s Esri User Conference, USAA demonstrated the use of deep learning capabilities in ArcGIS to perform automated damage assessment of homes after the devastating Woolsey fires. This is a critical task in damage claim processing, and using deep learning can speed up the process and make it more efficient. This blog post shows how this was done, from training the deep learning model to detecting damaged homes using it.
Damage assessment using drone imagery and deep learning
The first thing that deep learning needs is training data to look at and learn from. To build the model, we need to teach it what a damaged home looks like in drone imagery. For this, we used ArcGIS Pro to manually label a few hundred homes, marking them as damaged or undamaged by looking at their drone imagery. A GIS layer of the existing building footprints was used for this. We created a new field in the building footprints layer and gave an attribute of damaged or undamaged.
Exporting training samples
These were used to export training samples in the ‘Labelled Tiles’ format using the Export Training Data for Deep Learning tool in ArcGIS Pro. Pro has recently added support for exporting training data in the Labelled Tiles format to support this task.
Training the Damage Classifier model
Once we had the training data ready, we used ArcGIS Notebooks and the arcgis.learn module in the Python API to train the model. Essentially, all it took was about 10–12 lines of boilerplate arcgis.learn code that looked like this:
ArcGIS Notebooks is pre-configured with the necessary deep learning libraries, so no extra setup was required. Training the model using a ResNet34 architecture gave over 99% accuracy. This isn’t surprising, as deep learning is especially good at image classification tasks like these.
Inferencing
By the time the notebook completed running, it had already trained the model and updated the layer classifying all available building footprints as damaged or undamaged. A web map with results of the damage classification is shared here.
It turned out that due to newer developments, the building footprint layer that we had available was outdated — it did not contain all the buildings. Getting high quality geospatial data like this is labor intensive and costly. Could we use deep learning to extract updated building footprints from imagery? It turned out that we could! See how we did that in the next section.
Extracting building footprints
We used the existing building footprints as training data to train another deep learning model for extracting building footprints. This time, the model that we had to train was a pixel classification model, wherein each pixel belonging to a building needed to be labelled as such.
Exporting training data
ArcGIS Pro has support for exporting training samples in the Classified Tiles format for training such models. All we had to do was convert the existing building footprint layer to a classified raster and feed it into the tool.
Training the model
Training the model was straightforward. Our training notebook was just like the one used earlier, but this time we used a U-Net model in arcgis.learn for pixel classification.
The model trained here was used to segment out the pixels belonging to buildings. ArcGIS Pro readily supports models saved by arcgis.learn.
Extracting building footprints
We used Classify pixels using deep learning tool to segment the imagery using the model and post-processed the resulting raster in ArcGIS Pro to extract building footprints. To post process the classified imagery we used Model Builder — it works great for repetitive workflows, especially when we are experimenting with parameters to get the best results.
The results of the building footprint model are a treat to see:
A web map with the extracted building footprints is shared here. Having such high quality building footprints generated automatically with no human intervention is a testament to the progress AI has made. Doing this end-to-end workflow in ArcGIS made the process easy and straightforward with all the necessary tooling in place.
Inferencing
Finally, for extracting the updated building footprints in our entire area of interest covering over 60,000 buildings, we used ArcGIS Enterprise with distributed Raster Analytics capability and scaled out the processing so it could be done within 20 minutes — that’s a rate of generating 3,000 building footprints every minute!
Time is a critical resource, be it for processing insurance claims, or for providing aid to those affected by natural disasters like hurricanes, floods or fire. Applying deep learning techniques like these to assist with damage assessment after such unfortunate events can indeed be a life saver.
