NAFNET : A Machine Learning Model to Deblur Images

This is an introduction to「NAFNET」, a machine learning model that can be used with ailia SDK. You can easily use this model to create AI applications using ailia SDK as well as many other ready-to-use ailia MODELS.

Overview

NAFNET is a model that became the baseline for image restoration, published in April 2022 by MEGVII Technology. It is capable of deblurring and denoising images.

Blurred image (Source: https://github.com/megvii-research/NAFNet/blob/main/demo/denoise_img.png)

Model output (Source: https://github.com/megvii-research/NAFNet/blob/main/demo/denoise_img.png)

Simple Baselines for Image Restoration

GitHub — megvii-research/NAFNet: The state-of-the-art image restoration model without nonlinear…

Architecture

NAFNET offers a computationally efficient and simple method for establishing a baseline for image restoration.

To achieve this, it introduces a Nonlinear Activation Free Network (NAFNet) by eliminating nonlinear activation functions such as ReLU and GELU. This allows it to deliver high performance at half the computational cost of traditional methods.

Since Batch Normalization can become unstable with small batch sizes, it uses Layer Normalization, which is commonly used in Transformers.

Regarding the model architecture, it replaces GELU, which was traditionally used, with a simple multiplication combined with Layer Normalization.

NAFNET architecture (Source: https://arxiv.org/abs/2204.04676)

The model structure uses UNet.

NAFNETのアーキテクチャ（出典：https://arxiv.org/abs/2204.04676）

The model was trained using a batch size of 64 and 400K epochs, with learning performed using random cropping.

Precision

The GoPro and REDS datasets were used for image deblurring, and the SIDD dataset was used for image noise reduction. NAFNET achieves higher image quality (PSNR) with fewer operations (MACs) than previous technologies.

Below is a qualitative comparison of various image deblurring methods.

Benchmark (Source: https://arxiv.org/abs/2204.04676)

Same for image denoising methods.

Benchmark (Source: https://arxiv.org/abs/2204.04676)

Model variants

NAFNET offers multiple model variants trained on different datasets. The “width” indicates the number of layer blocks stacked, with 64 having a higher load but also higher performance than 32. Moreover, when using a model trained on the REDS dataset, JPEG noise is also removed.

Model variants (Source: https://github.com/megvii-research/NAFNet/tree/main)

Usage

You can use this model with ailia SDK using the following command to deblur images.

python3 nafnet.py --arch NAFNet-REDS-width64 -i input.jpg

The command below can be used for denoising.

python3 nafnet.py --arch NAFNet-SIDD-width32 -i input.jpg

You can also choose between variants to be used.

BLUR_LISTS = ['Baseline-GoPro-width32' ,'NAFNet-GoPro-width32', 'NAFNet-REDS-width64', 'Baseline-GoPro-width64','NAFNet-GoPro-width64']
NOISE_LISTS = ['Baseline-SIDD-width32', 'NAFNet-SIDD-width64', 'Baseline-SIDD-width64' ,'NAFNet-SIDD-width32']

ailia-models/image_restoration/nafnet at master · axinc-ai/ailia-models

This model has been quite effective in extreme case such as vigorously moving an iPhone in front of a webcam.

python3 nafnet.py --arch NAFNet-REDS-width64 -i iphone.jpg

Input image

Output image

ax Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.