PaddleSeg: Highly Accurate Segmentation Model Using Hierarchical Attention

This is an introduction to「PaddleSeg」, a machine learning model that can be used with ailia SDK. You can easily use this model to create AI applications using ailia SDK as well as many other ready-to-use ailia MODELS.

Overview

PaddleSeg is a highly accurate segmentation model based on PaddlePaddle (Baidu) and release in May 2020. Its multi-scale attention system achieves state-of-the-art results on the Cityscapes dataset.

Source: https://github.com/PaddlePaddle/PaddleSeg/tree/release/2.3/contrib/CityscapesSOTA

Hierarchical Multi-Scale Attention for Semantic Segmentation

PaddleSeg/contrib/CityscapesSOTA at release/2.3 · PaddlePaddle/PaddleSeg

Architecture

In segmentation, multi-scale inference is often used. Images of different resolutions are input to the network and the results are averaged or combined based on maximum values.

PaddleSeg proposes a multi-scale approach coupled with an attention mechanism. This allows the network to learn which scale of image is preferable to use in each case and output better segmentation.

For example, in the example below, the thin pole disappears at a scale of 0.5x. At a scale of 2.0x, the poles is correctly segmented, but conversely, the accuracy of the road segmentation decreases. The attention mechanism is then used to combines those results and get the best segmentation result possible.

Source: https://arxiv.org/abs/2005.10821

Also, attention is done in a hierarchical manner. In explicit attention, the coefficients of the composition of the output of each segmentation are obtained directly. Explicit attention directly finds the coefficients for each scale and compose them to compute final result. By doing attention in a hierarchical manner, the number of coefficients to be computed can be reduced and training can be done faster.

Source: https://arxiv.org/abs/2005.10821

The backbone used for this model is HRNet_w48.

PaddleSeg has achieved state-of-the-art results on the Cityscapes data set.

Source: https://arxiv.org/abs/2005.10821

Usage

You can use PaddleSeg with ailia SDK with the following command. The segmentation is performed on image input.jpg , the output composite image output.jpg and the mask image output_mask.jpg are generated.

$ python3 paddleseg.py --input input.jpg --savepath output.jpg

ailia-models/image_segmentation/paddleseg at master · axinc-ai/ailia-models

ax Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.