Academics | Yann LeCun Latest research: can help Gan to use discrete data arae

AI Technology Review News, Fair, head of the Facebook AI Research Center, Yann LeCun, one of the deep-learning troika, commented on a dynamic on Facebook yesterday. Yann LeCun forwarded the news from a New York University who worked with him, who was reading a doctoral student, Jake Zhao, introducing a paper that had been uploaded to ArXiv. This paper introduces a kind of "antagonistic regularization Automatic encoder" (arae,adversarially regularized autoencoders) which can help Gan to use discrete data. Yann LeCun, as one of the co-authors of this paper, forwarded the message and was interpreted and evaluated.

According to the original paper, AI Science and Technology Review compiled such a diagram, can be arae to solve the problem to do a general understanding:

As shown in the original dynamic and Yann LeCun forwarding comments, please see below

Yann LeCun's forwarding comments

Adversarially regularized autoencoders (adversarial regularization automatic encoder, Arae) is a new method that restricts the encoded content information to an automatic encoder and prevents the automatic encoder from learning the identity function. In order to achieve this goal, there have been some sparse ae (sparse automatic encoder), variational AE (variational automatic encoder, add noise in the code), Denoising AE (Noise reduction automatic encoder) Such an attempt, There is also a comparison between these comparisons and other methods based on MCMC (Markov Chain Monte Carlo, random simulations). This time, we use a set of adversarial generators and discriminator to generate a sample of the specified entropy, and we use the discriminator to make the coded distribution in the Automatic encoder match the distribution of the generator discriminator.

After training, the generator and encoder parts of this automatic encoder can be used as a generative model.

Arae can have a powerful usage, which is the ability to generate discrete structures (such as text) in a confrontational setting. Since the process of confrontation occurs on (continuous) encoding rather than output, this can alleviate the problems that the adversarial discriminator may encounter when it is used directly on discrete outputs.

The study was attended by New York University, Harvard University, Fair, several participants, a master of data science graduate from New York University, PhD in Reading Jake Zhao, Harvard University alumni, PhD in Reading Yoon Kim, New York University undergraduate Kelly Zhang, Harvard University student Sasha Rush, as well as myself. Jake Zhao's message content

Just published this article "Adversarially regularized autoencoders for generating discrete structures" (Auto encoder for the adversarial regularization of discrete structures), and I felt very excited, Several co-authors Yoon Kim, Kelly Zhang, Sasha Rush and Yann LeCun are also very powerful.

Paper Address: https://arxiv.org/abs/1706.04223

Code address (with torch and pytorch two versions): Https://github.com/jakezhaojb/ARAE

For adversarial training on discrete structures, we present a simple solution: train a sequence-to-sequence transformation model in a rebuilt manner, and then let Gan work in a contiguous coded space. At the same time, the gradient from Gan to the Sequence transformation model is helpful to train a meaningful automatic sequence conversion model. By effort, we got an image Gan network should have a set of ideal properties: Generative, Z-space interpolation, vector computing. We also make a quantitative evaluation of this method based on the quality of the sample generated by the language model.