How we improve polygonization with GAN networks

The rise of machine learning in remote sensing started rapidly with the invention of such great encoder-decoder semantic segmentation neural network architectures as U-Net, LinkedNet, DeepLab in 2015–2018. A lot of open-source and commercial projects successfully use such approaches.

With the output of such networks, you can easily count the number of buildings or calculate building area, however when it comes to Mapping — you need regular shapes.

Simplification — classic approach

If you look at any map, you will find that the majority of buildings could be approximated with rectangular shapes and most of the angles are right angles. By using these assumptions it is possible to make a simplification algorithm (aka “polygonization”), which will produce building shapes, similar to what could be seen in maps.

Decoder-encoder network output — outlines are pixel-approximated to the objects in satellite images

Unfortunately, decoder-encoder networks tend to produce masks with rounded edges and it makes simplification much harder. Some buildings just couldn’t be simplified this way.

Rounded edges output

If we look at the random area, which is quite large, there are usually 10%-15% of unsimplified buildings. Look at the example in the picture below.

Given one of our benchmarking areas in China which is supposed to be challenging for the segmentation networks. ~12% of all buildings haven’t been simplified

Simplification — GAN’s approach

It’s going to be a labor-intensive process and eventually will cost a lot if you try to train your network to produce clear edges at every single building. There could be another solution — mask refinement with the GAN network.

The GAN network is trained on the pairs of features that are polygonized correctly. Actually, when it comes to polygon refinement with the GAN network — the new network is applied to the selected features that haven’t been polygonized (in our processing results they can be extracted by the NULL value of “Shape_type” attribute), and tries to make them more “rectangular”. It makes simplification easier, but it does not significantly affect the metrics.

GAN training pipeline

Segmentation mask, refined segmentation mask, simplified refined segmentation mask

With this approach, we reduced the number of unsimplified buildings by more than 24 times — from 12% to 0.5%.

Fraction of unsimplified polygons: yellow — original, red — new

China benchmark with additional GAN’s “polygonization”

Implementation into Mapflow.AI

Following this approach we built another image processing pipeline to be implemented into Mapflow AI-platform.

The pipeline has got more stages, however, because of the use of selected polygons only it adds about 10% of overheads to the overall simplification processing time.

The new “simplification” can be applied to default building model — you can try it on Mapflow.ai — don’t forget to check the “Simplification” option. Here you go:

Mapflow.ai — login here to run processing in your area of interest

And of course, while GAN can fix minor edge distortions, some contours are beyond any repair.

Try it on http://mapflow.ai .

And tell us what you think about the new method and how it performs in your area of interest.