Image for post
Image for post

Style Transferring Of Image Using (CycleGAN)

Manish Nayak
May 11, 2019 · 6 min read

Introduction

Cycle-Consistent Adversarial Networks CycleGANs were proposed by Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. CycleGANs are a novel approach for translating an image from a source domain A to a target domain B. One of the cool feature of CycleGANs is that it doesn’t require paired training data to produce stunning style transfer results.

In many style transfer applications, paired data is a required for the training.

CycleGAN is doesn’t require paired data input to train a models.

A CycleGAN tries to learn a Generator network, which, learns two mappings. CycleGANs train two Generators and two Discriminators networks. which differs from most of the GANs with a single Generator and Discriminator network.

CycleGAN has two Generator networks

  • Generator A: Learns a mapping G:X ->Y, where X is an image from the source domain A and Y is an image from the target domain B. It takes an image from the source domain A, and converts it into an image that is similar to an image from the target domain B. Basically, the aim of the network is to learn a mapping so that G(X) is similar to Y.
  • Generator B: Learns a mapping F:Y->X, and then takes an image Y from the target domain B, and converts it into an image X that is similar to an image from the source domain A. Similarly, the aim of the network is to learn another mapping, so that F(G(X) is similar to X.

CycleGAN has two Discriminator networks

  • Discriminator A: The Discriminator A is to Discriminate between the images generated by the Generator network A, which are represented as G(X), and the real images from the source domain A, which are represented as X.
  • Discriminator B: The Discriminator B is to Discriminate between the images generated by the Generator network B, which are represented as F(Y), and the real images from the source domain B, which are represented as Y.

CycleGAN’s Architecture Diagram

The goal of CycleGANs is to learn how to map images from one domain A to another domain B. CycleGAN’s architecture diagram shows the use of two mapping functions G and F, and two Discriminators DX and DY.

  • The discriminator DX is used to verify the images from domain A and the translated images F(y).
  • The discriminator DY is used to verify the images from domain B and the translated images G(x).
  • The CycleGANs use a cycle consistency loss for image translation. The cycle consistency loss is used to determine whether an image X from domain A can be recovered from the translated image.

X to Y then reconstructed X CycleGAN’s Architecture

Y to X then reconstructed Y CycleGAN Architecture

Loss Functions

Like other GAN, CycleGANs also have a training objective function, which needs to minimize in training process. The loss function is a weighted sum of the following losses.

  • Adversarial loss.
  • Cycle consistency loss.

Adversarial loss :

It is a loss between the image from the real distribution domain A or domain B, and the images generated by the Generator networks. We have two mapping functions and we will be applying the adversarial loss to both of the mappings.

The adversarial loss for the mapping is

  • Where, X is an image from the distribution of domain A, and Y is an image from the distribution of domain B.
  • The Discriminator D(Y) try to find the difference between the generated image by the G mapping G(X), and the real image Y from a different distribution of domain B.
  • The Discriminator D(X) try to find the difference between the generated image by the F mapping F(Y) and the real image X from the distribution of domain A.
  • The objective of G is to minimize the adversarial loss function against a Discriminator D, which constantly tries to maximize it.

Cycle consistency loss:

A cycle consistent mapping function is a function that can translate an image x from domain A to another image y in domain B, and generate back the original image.

  • A forward cycle consistent mapping function appears as follows
  • A backward cycle consistent mapping function looks as follows
  • The formula for cycle consistency loss is as follows
  • With cycle consistency loss, the images reconstructed by F(G(x)) and G(F(y)) will be similar to x and y, respectively.

Full Objective Loss Functions

The complete objective function is a weighted sum of both the adversarial loss and the cycle consistency loss, which is represented as follows

Here, the first adversarial loss as follows

and second adversarial loss as follows

The first adversarial loss is calculated on the Generator A, and the Discriminator B. The second adversarial loss is calculated on the Generator B, and the Discriminator A.

Accompanied jupyter notebook for this post can be found here.

Conclusion

One of the more magical applications of CycleGANs is style transfer. here are list some really cool application of CycleGANs

  • Summer scenes converted into wintry ones, and vice versa.
  • Monet paintings converted into photo-realistic images.
  • Baskets of apples converted into baskets of oranges, and vice versa.
  • A photo of a horse converted into a zebra , and vice versa.
  • A video of a horse running in a field converted into a zebra.
  • Real photo to an artist’s painting.
  • A video of a drive taken during the day converted into a nighttime one.
  • Flat, low-quality photos converted into what appear to be ones captured by high-end single-lens reflex cameras.
  • Satellite image to map.
  • Face image to emoji, caricature or anime.
  • Body image to the avatar.
  • Colorization of grayscale photos.
  • Medical scan to a real photo.
  • Real photo to an artist’s painting.

I hope this article helped you get started building your own CycleGANs. I think it will at least provides a good explanation and understanding about CycleGANs.

Data Driven Investor

empowering you with data, knowledge, and expertise

Sign up for DDIntel

By Data Driven Investor

In each issue we share the best stories from the Data-Driven Investor's expert community. Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Manish Nayak

Written by

Machine Learning, AI & Deep Learning Enthusiasts

Data Driven Investor

empowering you with data, knowledge, and expertise

Manish Nayak

Written by

Machine Learning, AI & Deep Learning Enthusiasts

Data Driven Investor

empowering you with data, knowledge, and expertise

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store