Image for post
Image for post

Estimating Street Safeness after an Earthquake with Deep Learning

Is it possible to estimate with minimum expert knowledge if your street will be safer than others when an earthquake occurs?

Sergio Ramírez
Feb 4, 2020 · 8 min read

A question we answered with computer vision and collaboration. This work is part of an Omdena challenge on earthquake aftermath management.

Why?

The last devastating earthquake in Turkey occurred in 1999 ( on the Richter scale) around 150–200 kilometers from Istanbul. Scientists believe that this time the earthquake will burst directly in the city and the magnitude is predicted to be similar. A disaster might be coming…

This is the main motivation behind this hosted by

This along with the opportunity to test my ML engineer skills was enough for me to join the challenge.

I wanted to create a functional and creative solution far from those enveloped problems found in Kaggle or similar competition-based platforms.

Defining the problem, designing a working solution from scratch, and struggling with acquiring/pre-processing data are skills relatively difficult to acquire, and rare to find out there.

What’s the problem? Children need their parents!

After some days discussing with the hosts, collaborators, and the Omdena team about how to better prepare the city of for an upcoming disaster, we spotted a problem quite simple but at the same time really important for families: get reunited ASAP in earthquake aftermath!

Our target was set to provide safe and fast route planning for families, considering not only time factors but also broken bridges, landing debris, and other obstacles usually found in these scenarios.

Image for post
Image for post
Fatih, one of the most popular and crowded districts in Istanbul. Source: Mapbox API

We resorted to work on two tasks: creating a that would depict how dangerous is a particular area on the map, and a providing the safest and shortest path from A to B. The latter algorithm would rely on the previous heatmap to estimate safeness.

Challenge started! Deep Learning for earthquake aftermath management.

Image for post
Image for post
Source: Unsplash @loic

By this time, we optimistically trusted in open data to successfully address our problem. However, we realized soon that data describing buildings quality, soil composition, as well as pre and post-disaster imagery, were complex to model, integrate, when possible to find.

Bridges over streets, buildings height, 1000 types of soil, and eventually, interaction among all of them… Too many factors to control! So we just focused on delivering something more approximated.

Computer vision + Deep Learning is the answer

The question was: how to accurately estimate street safeness in Istanbul without such a myriad of data? What if we could roughly estimate path safeness by embracing distance-to-buildings as a safety proxy. The farther the buildings the safer the pathway.

For that crazy idea, firstly we needed buildings footprints laid on the map. Some people suggested borrowing buildings footprints from , one of the most popular open-source map providers. However, we noticed soon Open Street Map, though quite complete, has some blank areas in terms of buildings metadata which were relevant for our task. Footprints were also inaccurately laid out on the map sometimes.

Image for post
Image for post
Haznedar area (Istanbul). Source: Satellite image from Google Maps.
Image for post
Image for post
Haznedar area too, but few footprints shown. Blue boxes depict building footprints. Source: OpenStreetMap.

Big problem but… to the rescue! Using Deep Learning, we could rely on satellite imagery to detect where buildings were located and then, estimated closeness from pathways to them.

The next stone on the road was to obtain high-resolution imagery of Istanbul. With enough resolution to allow a ML model locates building footprints in the map as a standard-visually-agile human does. Likewise, we would also need some annotated footprints on these images so that our model can gracefully train.

First step: Building a detection model with PyTorch and fast.ai

Image for post
Image for post
SpaceNet dataset covering area for Rio de Janeiro. Source: https://spacenetchallenge.github.io/

Instead of labeling hundreds of square meters manually, we trusted on (and in particular, images for Rio de Janeiro) as our annotated data provider. This dataset contains high-resolution satellite images and building footprints, nicely pre-processed and organized which were used in a recent competition.

We used a Dynamic Unet model with an ImageNet pre-trained resnet34 encoder as a starting point for our model. This state-of-the-art architecture uses by default many advanced deep learning techniques, such as a one-cycle learning schedule or AdamW optimizer.

All these fancy advances in just a few lines of code.

Image for post
Image for post
fastai fancy plot advising you about learning rates.

We set up a balanced combination of Focal Loss and Dice Loss, and accuracy and dice metrics as performance evaluators. After several frozen and unfrozen steps in our model, we came up with good-enough predictions for the next step.

For more information about working with geospatial data and tools with fast.ai, please refer to [1].

Where is my high-res imagery? Collecting Istanbul imagery for prediction.

Finding high-resolution imagery was the key to our model and at the same time a humongous stone hindering our path to victory.

For the training stage, it was easy to elude the manual annotation and data collection process thanks to SpaceNet, yet during prediction, obtaining high-res imagery for Istanbul was the only way.

Image for post
Image for post
Mapbox sexy logo

Thankfully, we stumble upon Mapbox and its easy-peasy almost-free download API which provides high-res all over the world, and with different zoom levels. Slippy map tiles are 256 × 256 pixel files, described by x, y, z coordinates, where x and y represent 2D coordinates in the Mercator projection, and z the zoom level applied on earth globe. We chose a zoom level equal to 18 where each pixel links to real 0.596 meters.

Image for post
Image for post
Slippy map tiles on the Mercator projection (zoom level 2). Source: http://troybrant.net/blog/2010/01/mkmapview-and-zoom-levels-a-visual-guide/

As they mentioned on their webpage, they have a generous free tier that allows you to download up to 750,000 raster tiles a month for free. Enough for us as we wanted to grab tiles for a couple of districts.

Image for post
Image for post
Slippy raster tile at zoom level 18 (Fatih, Istanbul).

Time to predict: Create a mosaic like your favorite painter

Once all required tiles were stealing space from my Google Drive, it was time to switch on our deep learning model and generate prediction footprints for each tile.

Image for post
Image for post
Model’s prediction for some tile in Rio: sometimes predictions looked better than actual footprints.

Then, we geo-referenced the tiles by translating from the Mercator coordinates to the latitude-longitude tuple (that used by mighty explorers). Geo-referencing tiles was a required step to create our prediction piece of art with GDAL software.

Python snippet to translate from Mercator coordinates to latitude and longitude.

Concretely, gdal_merge.py command allows us to glue tiles by using embedded geo-coordinates in TIFF images. After some math, and computing time… voilà! Our high-res prediction map for the district is ready.

Image for post
Image for post
Raw predictions overlaid on Fatih. From lower degree of building presence confidence (blue) to higher (yellow).

Inverse distance heatmap

Building detection was not enough for our task. We should determine distance from a given position in the map to the closest building around so that a person in this place could know how safe is going to be to cross this street. The larger the distance the safer, remember?

The path-finding team would overlay the heatmap below on his graph-based schema and by intersecting graph edges (streets) with heatmap pixels (user positions), they could calculate the average distance for each pixel on the edge and thus obtaining a safeness estimation for each street. This would be our input when finding the best A-B path.

Image for post
Image for post
Distance-to-buildings heatmap in meters. Each pixel represents the distance from each point to the closest building predicted by our model. Blue means danger, yellow-green safeness.

Clue: computing distance pixel-building for each tile independently is sub-optimal (narrow view), whereas the same computation on the entire mosaic will render as extremely expensive (3.5M of pixels multiplied thousands of buildings).

Working directly on the mosaic with a sliding window was the answer. Thus, for each pixel (x,y), a square matrix composed by (x-pad, y-pad, x+pad, y+pad) pixels from the original plot is created. Pad indicates the window side length in the number of pixels.

Image for post
Image for post
Pixel-wise distance computation. Orange is the point, blue is the closest building around. Side length = 100 pixels.

If a pixel belongs to some building, it returns zero. If not, return the minimum euclidean distance from the center point to the building's pixels. This process along with numpy optimizations was the key to mitigate the quadratic complexity beneath this computation.

Repeat the process for each pixel and the safeness map comes up.

Image for post
Image for post
Distance heatmap overlaid on the satellite image. Blue means danger, yellow-green safeness.

Final words

My main takeaways from this challenge have been the possibility of defining a compelling solution for a huge problem in a bottom-up way (ruling with no bosses), as well as cooking the dataset by ourselves.

This is sometimes obviated in other ML competitions or challenges.

Especially rewarding was to see how my model was integrated with other ML components made by my mates. All of them re-assembled to satisfy a real necessity: get parents and children reunited in earthquake aftermath.

Thanks, for giving me this opportunity of sharpening my ML skills on a problem that matters.

References

[1] Dave Luo, How to Segment Buildings on Drone Imagery with Fast.ai & Cloud-Native GeoData Tools (2019), Medium.

Omdena

Building Real-World AI Solutions Collaboratively

Sergio Ramírez

Written by

PhD. & ML engineer focused on AI. Data gooder, and wargamer.

Omdena

Omdena

Omdena is a collaborative platform for building AI solutions to real-world problems through the power of bottom up collaboration.

Sergio Ramírez

Written by

PhD. & ML engineer focused on AI. Data gooder, and wargamer.

Omdena

Omdena

Omdena is a collaborative platform for building AI solutions to real-world problems through the power of bottom up collaboration.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store