Hack for good — Damage classification with drone images

This weekend we participated in the Hackathon for good event at The Hague. We teamed up with Eddie, Joanna and Lora. It was a super random team, we met each other at meetups and hackathons and we turned out to be a great team. Eddie is a super extrovert developer/influencer, Lora is an awesome technical product owner, and Joanna is a “solve anything” data consultant.

The challenge

We chose the Red Cross challenge, which was to classify damage of buildings based on their aerial images taken by drones and building outline geo data of St Martin. We decided to use a mix of tech:

  • Python for image processing

None of us had any experience working with image processing so it was quite a big challenge. We flew in on Thursday night already so we could prepare before the hackathon on Saturday.

Processing satellite images

The first obstacle was that images were huge (a few hundred megabytes). We could not open them in any browser. They were TIF images that include geo coordinates. After a bit of research we decided to use rasterio with python that can manipulate and crop these images for us.

Resulting image:

Working with geojson data

Geojson is a type of dataset where the form of an object (like a building) is described by its boundary’s coordinates.

An example for a geojson dataset:

To display geojson we used geopandas, which reads the json dataset, returns a pandas dataframe and converts the coordinate data to a geometric dataset.

Combining Tif with geojson

We decided that to train our model we wanted to have a cropped image of all buildings that were pre-classified as:

  • No damage

We had to find a way to crop the drone aerial images and find the buildings.

This resulted in a group of building images:

Applying filters

We then wanted to apply some filters so the model can better understand the images. We used equalization and edge detection to transform images.

We realized that not damage buildings have a more condensed pixel frequency and clearer edges.

We used scikit-image to manipulate images.

Not damaged (left) vs Destroyed (right)

Training model

We then uploaded all the images into customvision.ai and trained the model (sorry but none of us has experience with ML, I promise next time we will build our own model).

We got a result of almost 80% precision.

CustomVision was very easy to use, but it’s a blackbox so we would need to read a bit more about how it was trained and what was the neural network architecture.

The event

This was one of the best hackathons I’ve been to. The organisation was great, the datasets were amazing and we all had a lot of fun. We had fun activities, massage, awesome healthy food, drinks and sleeping place provided.

Things that didn’t go well

  • Using kaggle was not that easy: We ran out of the max 500 output file limit (we had 7000 building images)

Things that went well

  • I am so happy I could get to know the people in my team, I look up on all of them and respect them for their achievements

Resources

Thesis of our mentor from Red Cross, Daniel Kersbergen.

Harvard study about building damage classification

Data & Blockchain Enthusiast