The characterization of the language of "Wang Cao's related thesis accumulation in NLP"

Note: This article is not the author's notes, is the author usually see the public number of the paper push and introduction (such as paperweekly, hit Scir, etc.), feel good, have the accuracy of the NLP related papers, will they copied in this article, so that after the need to review.

The paper is mainly related to natural language representation, such as the characterization of words, the representation of sentences, etc.

Source: Harbin Scir
Recommended by: Shijihao (Research interests: Social computing, smart Finance)
Thesis title: An efficient Framework for learning sentence representations
Author: Lajanugen Logeswaran, Honglak Lee
Source: ICLR 2018
The main relevance of the paper: sentence expression Learning
A brief comment: This paper presents a new method of sentence expression learning. Natural language can convey the same semantic information in different ways. Existing sentences indicate that learning models (such as skip-thought) have not only learned semantic information in this task, but also learned the expression that is not related to semantics. In this paper, as a point of entry, this article does not deliberately emphasize the expression, but instead of the decoding process of the Encoder-decoder model is replaced by the classification process, which effectively avoids the problem of high computational cost of spatial search on the whole thesaurus. In this paper, a simple and efficient framework is presented, which uses the non-annotated data to learn the expression of sentences, and this method is called Quick-thought (QT).
By comparing with the Skip-thought model, it can be found that even though the order of the main clause and the subordinate clause is different in the expression mode, the method of this paper still can recognize the similar semantic representation. In many of the tasks that need to understand semantic information, such as film reviews, product reviews, subject classification, and opinion polarity, the experimental results and training speed can be better than those of previous work.
Article Link:
Https://openreview.net/pdf?id=rJvJXZb0W

Source Code Link:
Https://github.com/lajanugen/S2V

Source: Hit Scir
recommended by: Luo column (Research direction: Affective analysis)
Thesis title: Learned in translation:contextualized Word Vectors
Author: Bryan McCann, James Bradbury, Caiming Xiong, Richard socher
Source: NIPS
Paper Main relevance: context-dependent word vectors, migration learning
Comments: In the computer vision collar Domains, "Pre-training (pretrained)" Weights and other parameters can be complex and diverse embedded in downstream models, such as pre-trained CNNs models in imagenet datasets have been used to initialize other related tasks. In the field of natural language processing, "pre-training" often uses only pre-trained word vectors, such as Word2vec, glove, and so on, for these word vectors, each word is isolated, almost without the context information in the sentence, it can be said that these pre-trained word vectors are shallow, and not very good use of "pre-training" advantage , such as wasting the contextual information of a word. In this paper, the author proposes a more in-depth "pre-training", that is, using a pre-trained encoder in the machine translation task to produce a new word vector cove with context. The authors found that applying cove to a number of tasks downstream can be a great boost, with the task of fine-grained sentiment analysis (SST-5 datasets) and Natural language inference (Snli datasets) reaching state of the art (2017).
Specifically, the author trained the encoder of a neural network model through the English-German machine translation task, and then to the context-based Vector Cove. Then, the regular word vectors, such as cove and glove, are combined to get input for specific tasks downstream, ultimately improving the performance of the task model.
Article link:
https://arxiv.org/abs/1708.00107
Source Link:
Https://github.com/salesforce/cove