Deep learning articles and code collections for text categorizationOriginal: franklearningmachine Machine Learning blog 4 days ago
[1] convolutional neural Networks for sentence classification
Yoon Kim
New York University
EMNLP 2014
http://www.aclweb.org/anthology/D14-1181
This article mainly uses CNN to classify sentences based on pre-trained word vectors. The authors find that using fine-tuning to learn task-related word vectors can enhance the model effect.
Examples of network structures such as the following
The statistics for each dataset are as follows
The results of each model are compared as follows
A
The effect of the channel on the model result is shown for example
Code address
Https://github.com/yoonkim/CNN_sentence (Theano)
HTTPS://GITHUB.COM/DENNYBRITZ/CNN-TEXT-CLASSIFICATION-TF (TensorFlow)
Https://github.com/harvardnlp/sent-conv-torch (Torch)
Related Research Group
http://nlp.seas.harvard.edu/(Harvard University)
I'm a split line.
[2] A convolutional neural Network for modelling sentences
Nal Kalchbrenner
University of Oxford
ACL 2014
http://www.aclweb.org/anthology/P14-1062
This paper presents a dynamic convolutional neural network and uses it for semantic modeling in sentences. The pooling operator in this network is a dynamic K-max pooling method, which is used for linear sequences. The network in this paper can deal with the variable-length sentences, and deduce the feature graph for the sentence, which can not only capture the short-distance relation, but also can capture the long-distance relation. In addition, the network does not rely on parse trees and can be used in any kind of language.
Examples of network structures such as the following
The comparison of the narrow convolution is as follows
The overall structure is as follows
The results of each model are compared as follows
The model results are shown for example below
Code address
Https://github.com/FredericGodin/DynamicCNN (Theano/lasagne)
I'm a split line.
[3] character-level convolutional Networks for Text classification
Xiang Zhang et al.
NIPS 2015
Https://papers.nips.cc/paper/5782-character-level-convolutional-networks-for-text-classification.pdf
This article mainly discusses the character-level convolutional neural networks.
Model structure examples such as the following
Convolution layer example below
Full Connection Layer example below
Data set statistics are as follows
The results of each model are compared as follows
Code address
Https://github.com/zhangxiangxiao/Crepe (Torch)
Https://github.com/mhjabreel/CharCNN (TensorFlow)
Https://github.com/srviest/char-cnn-text-classification-pytorch (Pytorch)
I'm a split line.
[4] Hierarchical Attention Networks for Document classification
Zichao Yang et al.
Carnegie Mellon University, Microsoft
Naacl-hlt 2016
http://www.aclweb.org/anthology/N16-1174
This article presents a hierarchical attention network for document classification. The hierarchical structure of the model can correspond to the hierarchical structure of the document, and the attention mechanism of the network includes two attention mechanisms of the word level and the sentence level, which helps to discover important content in the document.
The layered attention network structure is as follows
Data set statistics are as follows
The results of each method are compared as follows
Code address
Https://github.com/richliao/textClassifier (Keras)
Https://github.com/ematvey/hierarchical-attention-networks (TensorFlow)
Https://github.com/EdGENetworks/attention-networks-for-classification (Pytorch)
I'm a split line.
[5] Recurrent convolutional neural Networks for Text classification
Siwei Lai et al.
Chinese Academy of Sciences
AAAI 2015
https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/viewFile/9745/9552
This article presents a cyclic convolution neural network for text categorization without the need for manual design features. loop structures are used to capture contextual information, which can reduce noise relative to traditional window-based neural network methods. In this paper, the maximal value pooling method is used to automatically select the more important words in the text and make them more useful to the text classification.
Examples of network structures such as the following
Data set statistics are as follows
The results of each method are compared as follows
The context window size effect is shown for example below
Code address
Https://github.com/airalcorn2/Recurrent-Convolutional-Neural-Network-Text-Classifier (Keras)
I'm a split line.
[6] Very deep convolutional Networks for Text classification
Alexis Conneau et al.
Facebook AI
ACL 2017
http://www.aclweb.org/anthology/E17-1104
This article uses VDCNN to process text at the character level, and the convolution and pooling operators are small, that is, they depend on fewer units. In this paper, 29 convolution layers are used.
Sample and label examples such as the following
The network structure is as follows
Where the convolution block structure is as follows
The number of convolution layers corresponding to the convolution blocks is as follows
Data set statistics are as follows
The results of each method are compared as follows
Code address
Https://github.com/geduo15/Very-Deep-Convolutional-Networks-for-Natural-Language-Processing-in-tensorflow (TensorFlow)
Https://github.com/zonetrooper32/VDCNN (TensorFlow Keras)
I'm a split line.
[7] Do convolutional Networks need to is deep for Text classification?
Hoa T. Le et al.
LORIA
AAAI 2018
https://aaai.org/ocs/index.php/WS/AAAIW18/paper/viewFile/16578/15542
This article discusses the importance of the depth of convolutional networks in text categorization.
Example of a shallow and wide convolutional neural network
A densenet example of a character level is shown below
Dense Block example below
The results of each model are compared as follows
Data set statistics are as follows
Code address
Https://github.com/lethienhoa/Very-Deep-Convolutional-Networks-for-Natural-Language-Processing (TensorFlow)
Deep learning articles and code collections for text categorization