Fiery generation against the network (GAN), where exactly are you

"Online Live" "TensorFlow Real Combat" author, Ppmoney Big Data algorithm director Huang Wenjian teacher is about to start: "Using TensorFlow to create automatic game DQN model", this course focuses on deep-q Network and the implementation of its tensorflow. Click to register

Since 2014, when Ian Goodfellow put forward the concept of generating the anti-network (GAN), the generation of the network became a hot research hotspot for academia, and Yann LeCun called it "the most exciting idea of machine learning in the last decade". A simple introduction to generating a confrontation network is to train a generator (Generator, called g) to generate realistic samples from random noises or potential variables (latent Variable), while training a discriminator (discriminator, (d) To identify real data and generate data, both training, until a Nash equilibrium, the generator generated data and real samples are not different, the discriminator can not correctly distinguish between the generated data and real data. The structure of Gan is shown in Figure 1.

Figure 1 Generating the basic architecture of the network against

In the past two years, academia has put forward a variety of Gan variants, such as conditional generation countermeasure network (Cgan), Information generation countermeasure Network (Infogan) and deep convolution generation countermeasure network (Dcgan), as well as a paper from last year: Image-to-image translation With Conditional adversarial Nets, we can see that Gan has been introduced to a variety of previous deep neural network tasks, such as the restoration of the original image from the segmented image (upper left corner of the first pair), to color black and white pictures (upper right corner of the first pair), Color according to the texture map (the first pair in the lower right corner), in addition, Gan can also do image super-resolution, dynamic scene generation, and more about GAN application please see another blog deep learning in the field of computer vision advances.

Figure 2 images to image translation

To think carefully, these tasks are actually the traditional deep neural network can do, such as the self-encoder (Autoencodor) and convolution deconvolution architecture can do, we cannot help but want, gan compared to the traditional deep neural network, its advantages. Some time ago, I have been more confused, the Chinese can find the data, is Ian Goodfellow in the generation of anti-network (GAN) thesis concludes several points, as follows: The Advantage model only uses the reverse propagation, but does not need the Markov chain, training without the need to infer the implicit variables; Theoretically, as long as the differentiable function can be used to build D and G, because it can be combined with the deep neural network to do a depth-generated model; g's parameter updates do not come directly from the data sample, but instead use the reverse propagation from D (which is one of the most significant differences compared to traditional methods). The shortcomings can be interpreted poorly, and the distribution of the generated model PG (G) has no explicit expression. More difficult to train, D and G need a good synchronization, such as D update k times and G update once.

The above is just a simple explanation, fortunately, I found on the Quora two similar problems, GAN and other generation model compared to what advantages, this question only one person answer, fortunately, he is Ian Goodfellow,gan inventor, his signature on the Quora is "I invented generative adversarial networks". And the other question is what the pros and cons of Gans are. The Great God of conscience Goodfellow also answered. I translated his two answers as follows:

The original problem 1:what is the advantage of generative adversarial networks compared with other generative models.
The benefits of building against the network compared to other build models.

In comparison with all the other models, I think: from the actual results, Gan looks to produce a better generation of samples. The GAN framework can train any build network (in theory practice, it is difficult to use reinforcement learning to train generators with discrete outputs), and most other architectures require generators to have some form of function, just as the output layer must be Gaussian. In addition all other frameworks require that the generator is all non-0 weights (put Non-zero mass everywhere), however, Gans can learn a model that produces sample points only in places close to real data (neural network layer) (Gans can learn models That generate points is only on a thin manifold that goes near the data). There is no need to follow any kind of factor decomposition to design the model, and all generators and discriminator will work properly. The Gans produces a sample at a time, but Pixelrnns requires a pixel to produce a sample, compared to the Pixelrnn,gan generated sample. If the discriminator is well trained, the generator can perfectly learn the distribution of the training sample, compared to the Vae,gans without a variational lower bound. In other words, the Gans is gradual and consistent, but VAE is biased.
Compared to the deep Boltzmann machine, Gans does not have a variational lower bound, and there is no tricky sub-function, the sample is generated once, rather than repeating the application of Markov chain to generate. Compared to the Gsns,gans generated samples are generated once, rather than repeating the application of the Markov chain. Compared to the nice and real nve,gans, there is no limit to the size of the potential variable (the input value of the generator); To be honest, I think other methods are also great, and they have a corresponding advantage over Gans.

The original question 2:what is the pros and cons of using generative adversarial networks (a type of neural network).
What are the pros and cons of generating a confrontational network (a neural network)?

It was known that Facebook had developed a means of generating realistic-looking images via a neural network. They used "GAN" aka "Generative Adversarial Networks". Could This is applied generation of other things, such as audio waveform via RNN? Why or?
Facebook has developed a way to generate realistic images based on neural networks, and they use Gan, also called build-to-fight networks, that can be applied to other things, such as generating audio waveforms from RNN. Why.

Advantages Gans is a semi-supervised method of training classifiers, can refer to our Nips paper and corresponding code. When you don't have a lot of tagged training sets, you can simply use our code without making any changes, usually because you don't have too many tag samples. I have also recently successfully used this code to write a paper with the Google Brain Department on the privacy aspects of deep learning. Gans can produce samples faster than a completely obvious belief network (nade,pixelrnn,wavenet, etc.) because it does not need to generate different data in the sample sequence. Gans does not require Monte Carlo estimation to train the network, people often complain that Gans training is unstable and difficult to train, but they are much simpler than training a Boltzmann machine that relies on Monte Carlo estimation and logarithmic distribution functions. Because the Monte Carlo method does not work well in high-dimensional spaces, Boltzmann machines have never been extended to tasks like imgenet. Gans at least after training on imagenet can learn to paint some of the dog. Compared to the variational self-encoder, Gans does not introduce any deterministic bias (deterministic bias), and the Variational method introduces a deterministic bias, because they optimize the lower bound of the logarithmic likelihood, rather than the likelihood itself, which appears to cause the VAES generated instances to be more ambiguous than Gans. Compared to the non-linear ICA (Nice, Real nve, etc.), the Gans does not have any specific dimensions for the potential variables entered by the generator or requires the generator to be reversible. The process of generating instances compared to Boltzmann machines and Gsns,gans only needs to run the model once, rather than iterating many times in the form of a Markov chain.

inferior training gan needs to achieve Nash equilibrium, sometimes can be done by gradient descent method, sometimes do not. We haven't found a good way to achieve Nash equilibrium, so it's not stable to train gan compared to vae or PIXELRNN, but I think in practice it's much more stable than training Boltzmann machines. It is difficult to learn to generate discrete data, just like text. Compared to the Boltzmann machine, Gans is difficult to guess the other pixel value according to a pixel value, Gans is born to do one thing, that is to produce all the pixels at once, you can use the Bigan to fix this feature, it allows you to use the same as the Boltzmann machine to use the Gibbs sampling to guess the missing value.
I talked about the problem 20 minutes before the Berkeley class. Course links, please bring your own ladder ~

—————————————————————————————————–
Above is the translation of Ian Goodfellow's original answer, please correct me if there is an inappropriate translation.
In a recent paper on Gan-generated images, the authors suggest that Gan can achieve better generalization results than normal convolution deconvolution. It is not known here whether the discriminator D as a monitor, when the accuracy of the D is close to 50%, stop training, to prevent the generator to cross-fit.

Author: Shongliang, graduate student, Southeast University
Zebian: He Yongcan, Welcome to the field of artificial intelligence technology contributions, manuscripts, to the article error correction, please send mail to heyc@csdn.net, this article for the CSDN original submission, without the author's permission shall not be reproduced.

SDCC 2017 • Shanghai railway station will be on March 2017 17-19th, the three technical Summit 24 guests, gathered in the domestic first-line internet company Daniel, talk about operations, databases and architecture hot topics and technology hotspots, the lean Operations Sponsor & Excellent Technology CEO Wang Jingban, MongoDB Greater China Chief architect has highlighted law and Huawei Software API open platform architect Li Linfeng and so on-site. Before March 5, 80 percent discount on tickets, 5 people or more buy a reduction of 400 yuan, details click on the registration of participants