Deep learning articles and code collections for text categorization

Deep learning articles and code collections for text categorizationOriginal: franklearningmachine Machine Learning blog 4 days ago

[1] convolutional neural Networks for sentence classification

Yoon Kim

New York University

EMNLP 2014

http://www.aclweb.org/anthology/D14-1181

This article mainly uses CNN to classify sentences based on pre-trained word vectors. The authors find that using fine-tuning to learn task-related word vectors can enhance the model effect.

Examples of network structures such as the following

The statistics for each dataset are as follows

The results of each model are compared as follows

A

The effect of the channel on the model result is shown for example

Code address

Https://github.com/yoonkim/CNN_sentence (Theano)

HTTPS://GITHUB.COM/DENNYBRITZ/CNN-TEXT-CLASSIFICATION-TF (TensorFlow)

Https://github.com/harvardnlp/sent-conv-torch (Torch)

Related Research Group

http://nlp.seas.harvard.edu/(Harvard University)

I'm a split line.

[2] A convolutional neural Network for modelling sentences

Nal Kalchbrenner

University of Oxford

ACL 2014

http://www.aclweb.org/anthology/P14-1062

This paper presents a dynamic convolutional neural network and uses it for semantic modeling in sentences. The pooling operator in this network is a dynamic K-max pooling method, which is used for linear sequences. The network in this paper can deal with the variable-length sentences, and deduce the feature graph for the sentence, which can not only capture the short-distance relation, but also can capture the long-distance relation. In addition, the network does not rely on parse trees and can be used in any kind of language.

Examples of network structures such as the following

The comparison of the narrow convolution is as follows

The overall structure is as follows

The results of each model are compared as follows

The model results are shown for example below

Code address

Https://github.com/FredericGodin/DynamicCNN (Theano/lasagne)

I'm a split line.

[3] character-level convolutional Networks for Text classification

Xiang Zhang et al.

NIPS 2015

Https://papers.nips.cc/paper/5782-character-level-convolutional-networks-for-text-classification.pdf

This article mainly discusses the character-level convolutional neural networks.

Model structure examples such as the following

Convolution layer example below

Full Connection Layer example below

Data set statistics are as follows

The results of each model are compared as follows

Code address

Https://github.com/zhangxiangxiao/Crepe (Torch)

Https://github.com/mhjabreel/CharCNN (TensorFlow)

Https://github.com/srviest/char-cnn-text-classification-pytorch (Pytorch)

I'm a split line.

[4] Hierarchical Attention Networks for Document classification

Zichao Yang et al.

Carnegie Mellon University, Microsoft

Naacl-hlt 2016

http://www.aclweb.org/anthology/N16-1174

This article presents a hierarchical attention network for document classification. The hierarchical structure of the model can correspond to the hierarchical structure of the document, and the attention mechanism of the network includes two attention mechanisms of the word level and the sentence level, which helps to discover important content in the document.

The layered attention network structure is as follows

Data set statistics are as follows

The results of each method are compared as follows

Code address

Https://github.com/richliao/textClassifier (Keras)

Https://github.com/ematvey/hierarchical-attention-networks (TensorFlow)

Https://github.com/EdGENetworks/attention-networks-for-classification (Pytorch)

I'm a split line.

[5] Recurrent convolutional neural Networks for Text classification

Siwei Lai et al.

Chinese Academy of Sciences

AAAI 2015

https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/viewFile/9745/9552

This article presents a cyclic convolution neural network for text categorization without the need for manual design features. loop structures are used to capture contextual information, which can reduce noise relative to traditional window-based neural network methods. In this paper, the maximal value pooling method is used to automatically select the more important words in the text and make them more useful to the text classification.

Examples of network structures such as the following

Data set statistics are as follows

The results of each method are compared as follows

The context window size effect is shown for example below

Code address

Https://github.com/airalcorn2/Recurrent-Convolutional-Neural-Network-Text-Classifier (Keras)

I'm a split line.

[6] Very deep convolutional Networks for Text classification

Alexis Conneau et al.

Facebook AI

ACL 2017

http://www.aclweb.org/anthology/E17-1104

This article uses VDCNN to process text at the character level, and the convolution and pooling operators are small, that is, they depend on fewer units. In this paper, 29 convolution layers are used.

Sample and label examples such as the following

The network structure is as follows

Where the convolution block structure is as follows

The number of convolution layers corresponding to the convolution blocks is as follows

Data set statistics are as follows

The results of each method are compared as follows

Code address

Https://github.com/geduo15/Very-Deep-Convolutional-Networks-for-Natural-Language-Processing-in-tensorflow (TensorFlow)

Https://github.com/zonetrooper32/VDCNN (TensorFlow Keras)

I'm a split line.

[7] Do convolutional Networks need to is deep for Text classification?

Hoa T. Le et al.

LORIA

AAAI 2018

https://aaai.org/ocs/index.php/WS/AAAIW18/paper/viewFile/16578/15542

This article discusses the importance of the depth of convolutional networks in text categorization.

Example of a shallow and wide convolutional neural network

A densenet example of a character level is shown below

Dense Block example below

The results of each model are compared as follows

Data set statistics are as follows

Code address

Https://github.com/lethienhoa/Very-Deep-Convolutional-Networks-for-Natural-Language-Processing (TensorFlow)

Deep learning articles and code collections for text categorization