"TensorFlow learning" The simplest GAN realization __gan

1.GAN Basic Idea

Generation against network Gan (generative adversarial networks) is a generation model proposed by Goodfellow in 2014. The core idea of GAN comes from the Nash equilibrium of game theory. It is set to participate in the game as a generator (generator) and a discriminant (discriminator), the generator captures the potential distribution of real data samples and generates new data samples; The discriminant is a two classifier to determine whether the input is a real or a generated sample.
In order to win the game, the two players need to continuously optimize, each improve their ability to build and discriminant, the learning optimization process is to find a Nash equilibrium between the both. both the generator and the discriminant can be used to study the hot deep Neural network at present . 2.GAN Structure

The calculation flow and structure of Gan are shown in the figure.
Any differentiable function can be used to represent the generator and the discriminant of Gan, so we use the differentiable function D and G to represent the discriminant and the generator respectively, their input is real data x and random variable Z. G (z) is a sample of the Pdata P_{data} generated by G to conform as much as possible to the real data distribution. If the input from the discriminant is from the real data, the callout is 1. If the input sample is g (z), the callout is 0.
The goal of D here is to achieve a two classification of the data source: True (from the distribution of real data x) or pseudo (the pseudo data G (z) from the generator), the objective of G is to make the pseudo data G (z) generated by itself D (g (z)) consistent with the representation D (x) of the real data x on D. The Learning method of 3.GAN

is actually the Minimax game of the generator and the classifier:
The goal of D is to maximize V (d,g)
The goal of G is to minimize Max V (d,g) 4.GAN implementations

The next step is to implement a vanilla GAN, the generator and the discriminant are all two layers of neural networks. The dataset uses Mnist.

The result of the implementation is this:

Let's do it step-by-step. ~ 4.1 Import Data

Import TensorFlow as TF
import numpy as NP
import Matplotlib.pyplot as Plt
import matplotlib.gridspec as grids PEC
import os from
tensorflow.examples.tutorials.mnist import input_data

sess = tf. InteractiveSession ()

mb_size = 128 Z_dim = mnist =

input_data.read_data_sets ('.. /.. /mnist_data ', one_hot=true)

4.2 Declaring Variables

def weight_var (Shape, name): Return
    tf.get_variable (Name=name, Shape=shape, initializer= Tf.contrib.layers.xavier_initializer ())


def bias_var (Shape, name): Return
    tf.get_variable (Name=name, shape =shape, Initializer=tf.constant_initializer (0))


# discriminater net

X = Tf.placeholder (Tf.float32, shape=[ None, 784], name= ' X ')

d_w1 = Weight_var ([784, 128], ' D_W1 ')
d_b1 = Bias_var ([128], ' D_B1 ')

d_w2 = Weight_var ( [128, 1], ' d_w2 ')
d_b2 = Bias_var ([1], ' d_b2 ')


theta_d = [D_w1, d_w2, D_B1, d_b2]


# generator net

Z = t F.placeholder (Tf.float32, Shape=[none, MB], name= ' Z ')

g_w1 = Weight_var ([128], ' G_W1 ')
g_b1 = Bias_var (  [128], ' G_B1 ')

g_w2 = Weight_var ([128, 784], ' g_w2 ')
g_b2 = Bias_var ([784], ' g_b2 ')

theta_g = [G_w1, G_W2, G_B1, G_B2]

4.3 Definition Model

def generator (z):
    g_h1 = Tf.nn.relu (Tf.matmul (z, G_W1) + g_b1)
    G_log_prob = Tf.matmul (g_h1, g_w2) + g_b2
    G_PR OB = Tf.nn.sigmoid (g_log_prob) return

    g_prob


def discriminator (x):
    d_h1 = Tf.nn.relu (Tf.matmul (x, D_W1) + D _B1)
    d_logit = Tf.matmul (d_h1, d_w2) + d_b2 D_prob
    = tf.nn.sigmoid (d_logit) return
    d_prob, D_logit

g_ Sample = Generator (Z)
d_real, d_logit_real = discriminator (X)
d_fake, D_logit_fake = discriminator (g_sample)

4.4 Setting loss function

The explanations in the paper are as follows:

Since TensorFlow can only do Minimize,loss function can be written as follows:

D_loss =-tf.reduce_mean (Tf.log (d_real) + tf.log (1.-D_fake))
G_loss =-tf.reduce_mean (Tf.log (D_fake))

Notably, the paper mentions that maximizing Tf.reduce_mean (Tf.log (D_fake)) is better than minimizing Tf.reduce_mean (1-tf.log (D_fake)).

Another way to do this is to take advantage of TensorFlow's own tf.nn.sigmoid_cross_entropy_with_logits function:

D_loss_real = Tf.reduce_mean (Tf.nn.sigmoid_cross_entropy_with_logits (
    logits=d_logit_real, Labels=tf.ones_ Like (d_logit_real))
D_loss_fake = Tf.reduce_mean (tf.nn.sigmoid_cross_entropy_with_logits (
    logits=d_ Logit_fake, Labels=tf.zeros_like (d_logit_fake)))
D_loss = d_loss_real + d_loss_fake
g_loss = Tf.reduce_mean ( Tf.nn.sigmoid_cross_entropy_with_logits (
    logits=d_logit_fake, Labels=tf.ones_like (D_logit_fake))

4.5 Optimization

D_optimizer = Tf.train.AdamOptimizer (). Minimize (D_loss, var_list=theta_d)
G_optimizer = Tf.train.AdamOptimizer ( ). Minimize (G_loss, Var_list=theta_g)

4.6 Training

Sess.run (Tf.global_variables_initializer ())
for it in Range (1000000):
   x_mb, _ = Mnist.train.next_batch (mb_ Size)

    _, D_loss_curr = Sess.run ([D_optimizer, D_loss], feed_dict={
                              x:x_mb, Z:sample_z (Mb_size, Z_dim)})
    _ , G_loss_curr = Sess.run ([G_optimizer, G_loss], feed_dict={
                              z:sample_z (mb_size, Z_dim)})

    if it% 1000 = 0:
        p Rint (' Iter: {} '. Format (IT))
        print (' D loss: {:. 4} '. Format (d_loss_curr))
        print (' G_loss: {:. 4} '. Format (g_loss _curr))
        print ()

4.7 Save the generated picture

In the 4.6 Training code add a paragraph, each 1000 step to save 16 pictures to generate:

Def sample_z (M, N): "Uniform prior for G (Z)" "Return Np.random.uniform ( -1., 1., Size=[m, N]) def plot (samples ): Fig = Plt.figure (figsize= (4, 4)) GS = Gridspec. 
        Gridspec (4, 4) gs.update (wspace=0.05, hspace=0.05) for I, sample in Enumerate (samples): # [i,samples[i]] imax=16
        Ax = Plt.subplot (Gs[i]) plt.axis (' Off ') ax.set_xticklabels ([]) ax.set_aspect (' equal ') Plt.imshow (sample.reshape), cmap= ' Greys_r ') return to fig if not os.path.exists (' out/'): Os.makedirs ( ' out/') Sess.run (Tf.global_variables_initializer ()) i = 0 for it in range (1000000): if it% 1000 = = 0:sampl Es = Sess.run (g_sample, feed_dict={z:sample_z (Z_dim)}) # 16*784 fig = Plot (Sampl

    ES) plt.savefig (' out/{}.png '. Format (str (i). Zfill (3)), bbox_inches= ' tight ') i + = 1 plt.close (fig) X_MB, _ = Mnist.train.next_batch (Mb_size) _, D_loss_curr = Sess.run ([d_optimizeR, D_loss], feed_dict={x:x_mb, Z:sample_z (Mb_size, Z_dim)}) _, G_loss_curr = Sess.run
        ([G_optimizer, G_loss], feed_dict={z:sample_z (Mb_size, Z_dim)}) if it% 1000 = 0: Print (' Iter: {} '. Format (it)) print (' D loss: {:. 4} '. Format (D_loss_curr)) print (' G_loss: {:. 4} '. Format
        (G_loss_curr)) Print ()

5 Reference

[1] http://wiseodd.github.io/techblog/2016/09/17/gan-tensorflow/
[2] Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing Systems. 2014.
[3] Wang Kunfeng, et al. "The research progress and prospect of generation against network GAN."