Deblur Photos Using Generic Pix2Pix

A Naive Approach with Minimal Domain Knowledge

Ceshine Lee
Sep 18, 2017 · 4 min read

Motivation / Background

Last week my partner came across a problem at work. There were some poorly shot photos that were quite blurry and needed to be repaired. Unsharp masking didn’t work well, along with a few free reparing softwares. The problem was solved by manually recreate important parts of the photo using Photoshop. But I couldn’t help but wonder if deblurring can be done via some generic deep learning algorithms.

I started with some super resolution algorithms, but soon realized that there are some differences. De-blurring, in essence, is trying to reverse convolution on an image (blind decovolution). Super-resolution, on the other hand, is trying to reverse the down-sampling on an image. Therefore I found pix2pix model should be more adequate for this task (as paired mappings between blurry photos and clear photos)[1]

(Warning: Restoring degrading images is a topic that has been actively researched, and on which I’m mostly ignorant. This post is about me throwing a generic algorithm at it. Please check the reference section for some examples of the research [2][3][4])

Implementation Overview

Explanation to Adversarial Training from [1]

The code is based on pix2pix implementation by mrzhu-cool on Github, with the following modifications.

  • The original GAN loss is replaced by Wasserstein loss (using a similar structure as in martinarjovsky/WassersteinGAN)[5].

The MIRFLICKR-25k dataset is used (in hope of generalizing better with real-life photos). The first 20k photos is used in training. Scaling and random cropping is applied. For the last 5k photos, around 2k are used as validation/development set, and the rest is reserved as test set (not used yet).

The artificial blurring is created by applying an uniform 3x3 filter and an Gaussian 5x5 filter (there’s a lot of rooms to be improved):

Left: Uniform Filter used by Blur(3); Right: Gaussian Filter used by GaussianBlur(5, 1.5)

Results — Uniform Filter

Achieved ~19.5 PSNR in development set with 200 epochs (batch size 16).

Samples from training dataset. Left: Blurred (Input); Middle: Recovered (Output); Right: Ground Truth
Real Photo Examples. Left: Blurred (Input); Middle: Recovered (Output)

Results — Gaussian Filter

Achieved ~19.8 PSNR in development set with 200 epochs (batch size 16).

Samples from training dataset. Left: Blurred (Input); Middle: Recovered (Output); Right: Ground Truth
Real Photo Examples. Left: Blurred (Input); Middle: Recovered (Output)

Remarks

  • The model performs reasonably well on the artificial dataset

Source Code

The code released on Github. It has been tested on ceshine/cuda-pytorch:0.2.0 Docker image. Please check the accompanying Dockerfile for details.


To-Do’s

  • Create a diversified blurring filters when training to make the model more generalized.

References:

  1. Isola, P., Zhu, J.-Y., Zhou, T., &Efros, A. A. (n.d.). Image-to-Image Translation with Conditional Adversarial Networks.

Machine Learning World

The best about Machine Learning, Computer Vision, Deep…

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store