Hierarchical Data Representation Model-multi-layer non-negative matrix decomposition

1 Overview

In order to understand complex data, we propose the concept of hierarchical feature extraction. One of the famous algorithms is the deep belief network (DBN) proposed by Hinton in 2006 ). With the success of deep structure training, many deep learning variants have been proposed. Although these multi-layer algorithms generate multi-layer methods to extract features and provide effective methods for complex problems, they do not tell us the relationship between features learned through multi-layer structures.

In this paper, we propose a hierarchical data representation model for multi-layer non-negative matrix decomposition. We provide a variant of the NMF algorithm, nsnmf for hierarchical learning. Here, we can intuitively display the hierarchical representation of features through the relationship between learning layers on the dataset. We have also proved that, unlike step-by-step learning, layered structures have learned more meaningful and helpful features with better distribution representation, it has better results in classification and small number of feature reconstruction, and reduces performance loss, even when representing low-latitude data.

2 non-smooth non-negative matrix decomposition (nsnmf)

The network structure consists of stack nsnmf. Nsnmf is a variant of NMF, which limits sparsity. NMF decomposition non-negative matrix X non-negative matrix W and H, where W is the feature, H is the corresponding coefficient or data representative; it is designed to reduce the error between the original data X and its reconstruction wh: c = 1/2 | X-WH | ^ 2. To limit the sparsity of the standard NMF, we introduce a sparse matrix: S = (1-θ) I (k) + θ/K ones (K ), where k is the number of features, θ is the smoothing coefficient, and the variation interval is between 0 and 1; I (k) is the unit matrix with the size of K * k, ones (k) is a matrix of K * K. We use s iteration to smooth the matrix. The closer θ is to 1, the smoother it will be. In terms of iterative updates, we use iterative s to smooth the H matrix, so h is so iterative: H = Sh. To compensate for the loss of sparsity, W is sparse.

3 Multi-Layer Structure

The proposed multi-layer NMF structure consists of multiple layers of multiple independent algorithms. We first train each layer separately. We

This article is from the squirrel blog, please be sure to keep this source http://apinetree.blog.51cto.com/714152/1560696

Hierarchical Data Representation Model-multi-layer non-negative matrix decomposition