Week 2 - Image Transformation according to Art Style
Hi everyone! We have done some research about our project steps, and today we are going to talk about these steps in more detail.
Style Clustering
In the first step, we want to eliminate less related arts from the dataset. For this purpose, we will apply some clustering methods to the image set. According to the Selective Clustering for Representative Paintings Selection research paper, it is observed that selective clustering improves the performance of the style transformation to eliminate outliers data. Selective clustering provides representative painting selection for one artist’s works. There are three other different clusterings that have been used to compare with representative painting selection methods which are K-means, A-cluster, S-cluster. Although the K-means clustering gives more accurate results for both average recall and precision, the selective clustering gives 91.42% accuracy for precision. So, we plan to use selective clustering in our model.
CycleGAN
For image-style transformation, the most proper model is Cycle-Consistent Adversarial Networks which is based on GAN (Generative Adversarial Network) architecture. The CycleGAN is a technique that involves the automatic training of image-to-image translation models without paired examples. The CycleGAN uses an additional extension to the GAN architecture called cycle consistency. This is the idea that an image output by the first generator could be used as input to the second generator and the output of the second generator should match the original image. Firstly, the image input is given to the GAN-1, and the output of the transformation is given to GAN-2 to get the input image. After, the cycle consistency loss calculates the difference between the image input to GAN-1 and the image output by GAN-2, and the model weights are updated accordingly to reduce the difference in the images.
We plan to compose these two different works in our project. Next week, we will have started the implementation part.
Stay tuned!
References:
- https://www.researchgate.net/publication/330989802_Selective_clustering_for_representative_paintings_selection
- https://arxiv.org/pdf/1703.10593v7.pdf
Contributors:
