Estimating Street Safeness after an Earthquake with Deep Learning
Is it possible to estimate with minimum expert knowledge if your street will be safer than others when an earthquake occurs?
A question we answered with computer vision and collaboration. This work is part of an Omdena challenge on earthquake aftermath management.
Why?
The last devastating earthquake in Turkey occurred in 1999 (>7 on the Richter scale) around 150–200 kilometers from Istanbul. Scientists believe that this time the earthquake will burst directly in the city and the magnitude is predicted to be similar. A disaster might be coming…
This is the main motivation behind this AI challenge hosted by Omdena and Impacthub Istanbul:
- Optimizing the Aftermath Management of Earthquakes with AI
This along with the opportunity to test my ML engineer skills was enough for me to join the challenge.
I wanted to create a functional and creative solution far from those enveloped problems found in Kaggle or similar competition-based platforms.
Defining the problem, designing a working solution from scratch, and struggling with acquiring/pre-processing data are extremely valuable skills relatively difficult to acquire, and rare to find out there.
What’s the problem? Children need their parents!
After some days discussing with the hosts, collaborators, and the Omdena team about how to better prepare the city of Istanbul for an upcoming disaster, we spotted a problem quite simple but at the same time really important for families: get reunited ASAP in earthquake aftermath!
Our target was set to provide safe and fast route planning for families, considering not only time factors but also broken bridges, landing debris, and other obstacles usually found in these scenarios.
We resorted to work on two tasks: creating a risk heatmap that would depict how dangerous is a particular area on the map, and a path-finding algorithm providing the safest and shortest path from A to B. The latter algorithm would rely on the previous heatmap to estimate safeness.
Challenge started! Deep Learning for earthquake aftermath management.
By this time, we optimistically trusted in open data to successfully address our problem. However, we realized soon that data describing buildings quality, soil composition, as well as pre and post-disaster imagery, were complex to model, integrate, when possible to find.
Bridges over streets, buildings height, 1000 types of soil, and eventually, interaction among all of them… Too many factors to control! So we just focused on delivering something more approximated.
Computer vision + Deep Learning is the answer
The question was: how to accurately estimate street safeness in Istanbul without such a myriad of data? What if we could roughly estimate path safeness by embracing distance-to-buildings as a safety proxy. The farther the buildings the safer the pathway.
For that crazy idea, firstly we needed buildings footprints laid on the map. Some people suggested borrowing buildings footprints from Open Street Map, one of the most popular open-source map providers. However, we noticed soon Open Street Map, though quite complete, has some blank areas in terms of buildings metadata which were relevant for our task. Footprints were also inaccurately laid out on the map sometimes.
Big problem but… Computer vision to the rescue! Using Deep Learning, we could rely on satellite imagery to detect where buildings were located and then, estimated closeness from pathways to them.
The next stone on the road was to obtain high-resolution imagery of Istanbul. With enough resolution to allow a ML model locates building footprints in the map as a standard-visually-agile human does. Likewise, we would also need some annotated footprints on these images so that our model can gracefully train.
First step: Building a detection model with PyTorch and fast.ai
Instead of labeling hundreds of square meters manually, we trusted on SpaceNet (and in particular, images for Rio de Janeiro) as our annotated data provider. This dataset contains high-resolution satellite images and building footprints, nicely pre-processed and organized which were used in a recent competition.
The modeling phase was really smooth thanks to fast.ai software.
We used a Dynamic Unet model with an ImageNet pre-trained resnet34 encoder as a starting point for our model. This state-of-the-art architecture uses by default many advanced deep learning techniques, such as a one-cycle learning schedule or AdamW optimizer.
All these fancy advances in just a few lines of code.
We set up a balanced combination of Focal Loss and Dice Loss, and accuracy and dice metrics as performance evaluators. After several frozen and unfrozen steps in our model, we came up with good-enough predictions for the next step.
For more information about working with geospatial data and tools with fast.ai, please refer to [1].
Where is my high-res imagery? Collecting Istanbul imagery for prediction.
Finding high-resolution imagery was the key to our model and at the same time a humongous stone hindering our path to victory.
For the training stage, it was easy to elude the manual annotation and data collection process thanks to SpaceNet, yet during prediction, obtaining high-res imagery for Istanbul was the only way.
Thankfully, we stumble upon Mapbox and its easy-peasy almost-free download API which provides high-res slippy map tiles all over the world, and with different zoom levels. Slippy map tiles are 256 × 256 pixel files, described by x, y, z coordinates, where x and y represent 2D coordinates in the Mercator projection, and z the zoom level applied on earth globe. We chose a zoom level equal to 18 where each pixel links to real 0.596 meters.
As they mentioned on their webpage, they have a generous free tier that allows you to download up to 750,000 raster tiles a month for free. Enough for us as we wanted to grab tiles for a couple of districts.
Time to predict: Create a mosaic like your favorite painter
Once all required tiles were stealing space from my Google Drive, it was time to switch on our deep learning model and generate prediction footprints for each tile.
Then, we geo-referenced the tiles by translating from the Mercator coordinates to the latitude-longitude tuple (that used by mighty explorers). Geo-referencing tiles was a required step to create our prediction piece of art with GDAL software.
Concretely, gdal_merge.py command allows us to glue tiles by using embedded geo-coordinates in TIFF images. After some math, and computing time… voilà! Our high-res prediction map for the district is ready.
Inverse distance heatmap
Ok, I see my house but should I go through this street?
Building detection was not enough for our task. We should determine distance from a given position in the map to the closest building around so that a person in this place could know how safe is going to be to cross this street. The larger the distance the safer, remember?
The path-finding team would overlay the heatmap below on his graph-based schema and by intersecting graph edges (streets) with heatmap pixels (user positions), they could calculate the average distance for each pixel on the edge and thus obtaining a safeness estimation for each street. This would be our input when finding the best A-B path.
But how to produce this picture from the raw prediction map? Clue: computing distance pixel-building for each tile independently is sub-optimal (narrow view), whereas the same computation on the entire mosaic will render as extremely expensive (3.5M of pixels multiplied thousands of buildings).
Working directly on the mosaic with a sliding window was the answer. Thus, for each pixel (x,y), a square matrix composed by (x-pad, y-pad, x+pad, y+pad) pixels from the original plot is created. Pad indicates the window side length in the number of pixels.
If a pixel belongs to some building, it returns zero. If not, return the minimum euclidean distance from the center point to the building's pixels. This process along with numpy optimizations was the key to mitigate the quadratic complexity beneath this computation.
Repeat the process for each pixel and the safeness map comes up.
Final words
My main takeaways from this challenge have been the possibility of defining a compelling solution for a huge problem in a bottom-up way (ruling with no bosses), as well as cooking the dataset by ourselves.
This is sometimes obviated in other ML competitions or challenges.
Especially rewarding was to see how my model was integrated with other ML components made by my mates. All of them re-assembled to satisfy a real necessity: get parents and children reunited in earthquake aftermath.
I don’t forget my teammates cooperating together like bees to come up with a working solution for a real problem. Frustration was flying around sometimes during the challenge, but it made the final reward tastier at the end.
Thanks, Omdena for giving me this opportunity of sharpening my ML skills on a problem that REALLY matters.
References
[1] Dave Luo, How to Segment Buildings on Drone Imagery with Fast.ai & Cloud-Native GeoData Tools (2019), Medium.
