From sketches to photo realistic images: an introduction to image synthesis
Hi there AI enthusiast! So glad you could join us! Here we gather the major terms in Artificial intelligence and try to define them as simple as possible. Today, we chose Image Synthesis as our next subject.
What if you could become an artist in less time than it would require, let’s say approximately 3 years? All you had to do is to make a simple sketch, press some buttons and then poof, you have created a piece of artwork. Does this sound Impossible? Not anymore.
Thanks to the great evolution of Computer Vision and all other cool algorithms that were developed afterward, AI is now slowly, but surely, learning how to design images, based on input. The process evolves with heavy steps, since the major problem with computers is the way they understand images, and also because human appreciation to what looks good is subjective and personal.
Now, let’s see a few of the algorithms that can help you turn your black and white sketch into a full image:
Sketch-based image retrieval
This can be achieved by a transformation of GoogLeNet architecture into a triplet network. Thus, it allows the network learn across both the sketch and the image domain, and also a shared feature space between the two.
Image synthesis with Neural Networks
Using deep Convolutional Neural Networks (dCNN) for more productive tasks, such as texture synthesis. They could be used with a VGG-19 network, to transform the texture features of the “style” image within each layer of the network into a set of Gram metrics whilst capturing high-level of the content image.
Another option is to use Generative Adversarial Networks. As we presented them in the past article, there are two networks, one that plays the role of creation and the other of the discriminator. Thus, if trained enough, they could be able to generate more accurate image than the dCNN one.
MRF-based image synthesis
This type of method implies the use of generative Markov random field models, where dCNNs would work both for photorealistic and non-photorealistic image synthesis. Instead of using Gram matrices, the MRF maintains local patterns of the specified style, while using the same VGG-19 network, but trained only on images.
So, as we can see, image synthesis still has a long way to go. Or not, varying on each one’s opinion. But AI makes great progresses and, maybe sooner than we expect, we could see a remake of Star Wars with puppies instead of Jedis. The entertainment industry will certainly have a bright future.
We will continue our series of cool AI-terms article next week with “Machine Learning”. Until then, drop us a line on Synaptech’s Facebook or Twitter. See you!
