Super Resolution is the goal of transforming low-resolution image to a high-resolution one, so we can try and uncover new information. Popularized by the TV show “CSI”, the defining “zoom and enhance” effect has seen alot of progress with recent deep learning approaches.
Dong et al proposed a deep convolution neural network, SRCNN, to learn upsampling from low-res to high-res images [1].
Shi et Al then introduced an efficient method, ESPCN, for super resolution by performing filters in low-resolution space, and then learning upsampling mapping [2].
This subpixel convolution could replace the need for deconvolution filters to perform upsampling. As a bonus, it also reduces the number of parameters the neural network has to learn.
An important side effect of subpixel convolution is that it alleviates the need for intense GPU machines for experimenting. Since the paper already compares image quality, let’s just compare CPU execution times on a macbook.
(NOTE: Not completely accurate, with overhead etc, but the speed difference is significant)
Dataset
- randomly selected 805 images from celebA dataset [3]
- Low-res: 72×72
- High-res: 288×288 (scale=4)
Experiment
- Network architectures
- SRCNN: 9–5–5 (Convolution 9×9, 5×5, 5×5)
- ESPCN: 5–3–3
- Stack
- Tensorflow 1.1.0
- Keras 2.0.2
Results
- SRCNN, 1epoch: 26.02 sec
- ESPCN, 1epoch: 2.23 sec
- SRCNN, 10epochs: 263.05 sec
- ESPCN, 10 epochs: 9.29 sec
A Keras implementation for subpixel convolution: https://github.com/twairball/keras-subpixel-conv
References
Originally published at lioniq.wordpress.com on June 28, 2017.
