Restricted Boltzmann Machine Learning (1)

Time: 2014.07.02

Location: Base

------------------------------------------------------------------------

I. Brief Introduction

9 RBM) is a type of random neural network model with two-layer structure, symmetric link without self-feedback. The layer and layer are fully connected, and there is no link in the layer, that is, a two-part diagram.

RBM is an effective feature extraction method. It is often used to initialize a feed-forward neural network and can significantly improve the generalization ability. A deep belief network consisting of multiple RBM structures can extract better and more abstract features for classification. Let's start with the Boltzmann Machine first, and then bring out the simplified version of the Boltzmann Machine-restricted Boltzmann Machine.

------------------------------------------------------------------------

2. Boltzmann Machine (BM)

Proposed by Hinton, the great god of 1986, the Boltzmann Machine is a random Neural Network rooted in statistical mechanics. Neurons in the network are random neurons. The output of neurons is in two states: inactive and inactive. The output is represented by binary 0 1. The value of the status is determined by the probability statistics method.

BM is a feedback neural network composed of full connections of random neurons. It is symmetric and has no self-feedback. It contains a visible layer and a hidden layer. As shown in:

BM has powerful unsupervised learning capabilities and is able to learn complex rules in data. The cost is that training (learning) takes a long time. In addition, it is not only difficult to accurately calculate the distribution represented by BM, it is also difficult to obtain a random sample that follows the distribution represented by BM. Therefore, a RBM restriction is introduced.

------------------------------------------------------------------------

3. Restricted Boltzmann Machine (RBM)

RBM has a visible layer, a hidden layer, no connection in the layer, and full connection between the layer and the layer. It is a bipartite graph with the following structure.

The structure of the restricted Boltzmann Machine is characterized by that the activation conditions of each hidden layer unit are independent when the visible layer unit state (input data) is given. In turn, when the hidden layer unit State is given, the activation conditions of the visible layer Unit are also independent. In this way, even though the distribution represented by RBM is still not effectively calculated, random samples that are subject to the distribution represented by RBM can be obtained through the Gaussian sampling. As long as the number of hidden layer units is sufficient, RBM can fit any discrete distribution. Hinton proposed a quick learning algorithm for RBM in 2002, and compared the contrastive divergence (CD) algorithm to encourage everyone to study and discuss RBM. In terms of application, the RBM model has been successfully used to solve different machine learning problems, such as classification, regression, dimensionality reduction, high-dimensional time series modeling, image feature extraction, and collaborative filtering.

In 2006, Hinton and others proposed deep belief nets (DBN) and provided an efficient learning algorithm for this model. It is the main framework of the current deep learning algorithm, in this algorithm, a DBN model is made up of several RBM stacks. The training process is trained from low to high layer by layer, which is described as follows:

1. The bottom RBM is trained based on raw input data.

2. Use the features extracted from the base RBM as the input of the top RBM to continue training.

3. repeat this process to train as many RBM layers as possible.

Because RBM can be quickly trained through CD, this framework bypasses the overall high complexity of DBN training, simplifying DBN training to training multiple RBM, thus simplifying the problem. In addition, after training in this way, you can use the traditional global learning algorithm, such as the BP algorithm, to fine-tune the network, so that the model converges to the local most advantageous.

In general, the whole process is equivalent to training RBM layer by layer, initializing model parameters to better values, and further training through a small number of traditional learning algorithms, that is, Wei tune, it not only solves the problem of slow model training speed, but also achieves good results. A large number of experiments show that this method can generate very good initial parameter values, greatly improving the modeling capability of the model. The DBN model with RBM as the basic structure is one of the most effective deep learning algorithms currently.