GANs in Tensorflow (Part II)

MNIST digits generated by the generator.

Welcome to the second post, we will try to code a toy GAN to generate MNIST digits which are very simple. Check out Introduction to GANs before going through this post.

MNIST, MNIST everywhere

So lets get started….

  1. Import all the dependencies
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

2. Import the data

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets(“/tmp/data/” , one_hot= True)

3. Define the parameters for network and training

You can play with the learning rate, hidden layer size and batch size parameters.

epoch           = 100000
batch_size = 128
learning_rate = 2e-4
img_size = 784 #Input Image vector (28x28)
gen_hidden_dim = 256 #Hidden Layer size
disc_hidden_dim = 256 #Hidden Layer size
noise_dim = 100

4. Define the initialization function.

Here we will use Glorot Initialization. For more details you can look up to this good article by Andy

def glorot_init(shape):
std_dev = 1./ tf.sqrt(shape[0]/2.)
return tf.random_normal(shape = shape , stddev = std_dev)
Architecture

5. Define the Generator Layer:

#generator Hidden Layer
gen_hidden = tf.Variable(glorot_init([noise_dim , gen_hidden_dim]))
gen_hidden_bias = tf.Variable(tf.constant(0.1,shape=[gen_hidden_dim]))
#Generator Output Layer (Need to output generated images)
gen_out = tf.Variable(glorot_init([gen_hidden_dim , img_size]))
gen_out_bias = tf.Variable(tf.constant(0.1,shape = [img_size]))

6. Define the Discriminator Layers:

#Hidden Layer of Discriminator
disc_hidden = tf.Variable(glorot_init([img_size , disc_hidden_dim]))
disc_hidden_bias = tf.Variable(tf.constant(0.1,shape = [disc_hidden_dim]))
#Output Layer of Discriminator (Binary Output for image belong to data or generator)
disc_out = tf.Variable(glorot_init([disc_hidden_dim,1]))
disc_out_bias = tf.Variable(tf.constant(0.1,shape = [1]))

7. Define the Generator architecture

def generator(z):
h1 = tf.nn.relu(tf.matmul(z,gen_hidden) + gen_hidden_bias)
out = tf.nn.sigmoid(tf.matmul(h1,gen_out) + gen_out_bias)
return out

8. Define the Discriminator architecture

def discriminator(x):
h1 = tf.nn.relu(tf.matmul(x , disc_hidden) + disc_hidden_bias)
out= tf.nn.sigmoid(tf.matmul(h1, disc_out) + disc_out_bias)
return out

9. Define the placeholder for Generator and Discriminator input

gen_input = tf.placeholder(tf.float32, shape=[None, noise_dim])
disc_in = tf.placeholder(tf.float32 , shape=[None , img_size])
gen_sample = generator(gen_input)
disc_real = discriminator(disc_in)
disc_fake = discriminator(gen_sample)

10. Define the Loss for Generator and Discriminator

gen_loss  = -tf.reduce_mean(tf.log(disc_fake))
disc_loss = -tf.reduce_mean(tf.log(disc_real) + tf.log(1. — disc_fake))

11. Define the Optimizer and training variables

optimizer_gen = tf.train.AdamOptimizer(learning_rate= learning_rate)
optimizer_disc= tf.train.AdamOptimizer(learning_rate= learning_rate)
gen_var = [gen_hidden , gen_out ,gen_hidden_bias , gen_out_bias]
disc_var= [disc_hidden , disc_out , disc_hidden_bias , disc_out_bias]
train_gen = optimizer_gen.minimize(gen_loss , var_list= gen_var)
train_disc = optimizer_disc.minimize(disc_loss , var_list = disc_var)

12. And Finally Train

init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
g_loss= []
d_loss=[]
for i in range(1,epoch):
batch_x , _ = mnist.train.next_batch(batch_size)
z = np.random.uniform(-1.,1., size= [batch_size , noise_dim])
# print (z.shape)
feed_dict = {disc_in: batch_x , gen_input : z}
_ , _ , g1 , d1 = sess.run([train_gen, train_disc , gen_loss, disc_loss], feed_dict= feed_dict)
g_loss.append(g1)
d_loss.append(d1)
if i % 1000 == 0:
print(‘Step %i Generator Loss : %f Discriminator Loss : %f ‘%(i,g1,d1))

13. Plot the Loss of Discriminator and Generator

plt.plot(d_loss ,label = ‘Discriminator Loss’)
plt.plot(g_loss ,label = ‘Generator Loss’)
plt.xlabel(‘Iterations’)
plt.ylabel(‘Loss’)
plt.legend()

14. Now Visualize the Data generated

n_image = 6 
pic = np.empty((28*n_image,28*n_image))
for i in range(n_image):
z = np.random.uniform(-1.,1.,size = [n_image, noise_dim])
g = sess.run(gen_sample , feed_dict= {gen_input : z})
for j in range(n_image):
pic[i*28:(i+1)*28,j*28:(j+1)*28] = g[j].reshape([28,28])

plt.figure(figsize = (n_image,n_image))
plt.imshow(canvas, origin =”upper” , cmap = “gray”)
plt.show()
Looks Good :)

Check out Quick Introduction to GANs

References :

  1. Aymeric Damien TensorFlow-Examples