week2 : sw4p_m0d3ls

Theme: Learning Based Image Colorization

Team Members: Bugrahan Akbulut, Gökay Atay

REMINDER

Previous week we tested pretrained deep model[1] on some sample images.

In the first row there are original images, second row contains predicted colors

Also we tested model an some anime images model failed at such cases.

Original Image — Colorized Image

After investigate testing of pretrained model to make some improvement we thought hard. And we end up two idea : the first one is changing backbone of colorization network with ResNet, second one is the expand of dataset with anime images.

ResNet presented by Microsoft Research team at 2015[2]. Deep convolutional neural networks have led to a series of breakthroughs for image classification. Many other visual recognition tasks have also greatly benefited from very deep models. So, over the years there is a trend to go more deeper, to solve more complex tasks and to also increase /improve the classification/recognition accuracy. But, as we go deeper; the training of neural network becomes difficult and also the accuracy starts saturating and then degrades also. Residual Learning tries to solve both these problems.

Overview of ResNet

What is Residual Learning?

In general, in a deep convolutional neural network, several layers are stacked and are trained to the task at hand. The network learns several low/mid/high level features at the end of its layers. In residual learning, instead of trying to learn some features, we try to learn some residual. Residual can be simply understood as subtraction of feature learned from input of that layer. ResNet does this using shortcut connections (directly connecting input of nth layer to some (n+x)th layer. It has proved that training this form of networks is easier than training simple deep convolutional neural networks and also the problem of degrading accuracy is resolved. This is the fundamental concept of ResNet.

We are using Caffe Framework to develop train/test for our model. We combined feature extraction from ResNet and probabilty distrubution from Colorful Image Colorization’s model.

We didn’t train model yet but there is a result from vanilla model:

We are planning train ResNet on a relatively small data set and see what it will returns. But we can not predict effect of Residual Blocks on learning such problem, according to the very first results we will examin our model again.

References

[1] R. Zhang, P. Isola, and A. A. Efros. Colorful image colorization. ECCV, 2016.

[2] He, K.; Zhang, X.; Ren, S.; and Sun, J. 2015. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385.