Gan (Generative Warfare network) __ Machine learning

The Generative Warfare network (GAN, generative adversarial Networks) is a deep learning model, which is one of the most promising methods for unsupervised learning in the complex distribution. The model produces quite good output through the mutual game Learning of the framework (at least) two modules: the generation model (generative models) and discriminant models (discriminative model). In the original GAN theory, G and D are not required to be neural networks, only if they can fit the corresponding function of generation and discrimination. However, in practical use, deep neural networks are used as G and D in general. An excellent GAN application requires a good training method, otherwise it may cause the output to be unsatisfactory due to the freedom of the neural network model. Chinese name generation against network foreign language name generative adversarial Networks abbreviation GAN domain machine learning/Computer vision Directory1 Development History 2 Method 3 Application ▪ image generation ▪ data enhancement Development HistoryThe editor, Ian J. Goodfellow, in October 2014, in generative adversarial networks, presented a new framework for generating models from the confrontation process estimation. In the framework, two models are trained at the same time: the generation model G that captures the data distribution, and the discriminant model D for estimating the probability of the sample from the training data. G's training program is to maximize the probability of a D error. This frame corresponds to a maximum set lower limit for both sides of the game. It can be proved that there is only one solution in the space of any function G and D, which makes G reproduce the training data distribution, and d=0.5. In cases where G and d are defined by a multilayer perceptron, the entire system can be trained in reverse communication. No Markov chains or an expanded approximate inference network are required during training or generation of samples. The experiment proved the potential of this framework by qualitative and quantitative evaluation of the generated samples [1]. Method The model of the editing machine learning can be divided into two types: the model of Generation (generative) and the discriminant (discriminative model). The discriminant model requires input variables, which are predicted by some kind of model. The generation model is given some kind of implied information to produce the observational data randomly. To give a simple example, discriminant model: Given a picture of whether the animal in this picture is a cat or a dog generate model: Give a series of pictures of cats, create a new cat (not in the dataset) for discriminant models, loss functions are easy to define because the output is relatively simple. But for the generation model, the definition of loss function is not so easy. Our expectation of generating results is often ambiguous and difficult to define in mathematical axiomatic form. So it is advisable to give the feedback part of the model to the Discriminant model processing. This is Goodfellow. He generative and discrimitive the two major classes of machine learning in the  [2]   . The basic principle of Gan is actually very simple, here to generate pictures as an example to illustrate. Suppose we have two networks, G (generator) and D (discriminator). As its name implies, their function is: g is a network that generates pictures, it receives a random noise Z, generates pictures through this noise, and makes G (z). D is a discriminant network to determine whether a picture is "real". Its input parameter is the x,x represents a picture, the output D (x) represents the probability that x is the real picture, if is 1, represents 100% is the real picture, but the output is 0, on behalf of cannot be the real picture. In the training process, the goal of generating network G is to generate real images to deceive the discriminant network D. and D's goal is to try to the G-generated pictures and real pictures separate. In this way, G and D form a dynamic "game process". What is the result of the last game? In the ideal state, G can generate a picture g (Z) that is sufficiently "genuine". For D, it is difficult to determine whether G-generated pictures are true, so D (g (z)) = 0.5. So our goal is to achieve: we have a generative model G, which can be used to generate pictures.  [3]   Goodfellow theoretically proves the convergence of the algorithm  [1]   , and when the model converges, the generated data has the same distribution as the real data (guaranteed the model effect). In the GAN formula, X represents the real picture, Z represents the noise of the input G network, G (z) represents the image generated by G network, D (·) The probability that the D network determines whether the picture is true or not. ApplicationEdit Image GenerationAt present, the most common use of Gan is image generation, such as Super Resolution task, semantic segmentation and so on. Data EnhancementUse the image produced by Gan to do data enhancement, as shown in the figure. The main problem is that for small data sets, the amount of data is insufficient, if you can generate some of the good. If Gan produces a picture. How to give these data label. Because they do not belong to the predefined categories compared to the original data. In [4], a number of attempts have been made. The experimental idea is also very simple, using the original data (even if only 2000 pictures) to train a Gan, and then generate pictures, add to the training set. To sum up: GAN generation data can be used on actual image problems (not just like mnist on the toy dataset work) The authors are promoted on two pedestrian recognition datasets and a fine-grained recognition data set for birds. GAN data has three ways to give pseudo label, assuming that we do five classification of the generated data as a new class, six classification, then the resulting image label can be (0, 0, 0, 0, 0, 1) this gives. According to the highest degree of confidence dynamic distribution, the probability of a higher probability of a high (0, 0, 1, 0, 0) since all classes are not, then you can refer to Inceptionv3, engage in label smooth, each type of confidence is the same (0.2, 0.2, 0.2, 0.2, 0.2) Note: The author wrote the code in December 16, when the Gan effect is not spicy good, with this effect is also possible, because the generation of samples are not very "true", so played a regular role.