Rb4.09.03 RBM

Essence of deep learningBy building machine learning models with many hidden layers and massive training data, we can learn more useful features and ultimately improve the accuracy of classification or prediction. Therefore, "deep model" is a means, and "feature learning" is a goal. Different from traditional shallow learning, deep learning has the following differences: 1) It emphasizes the depth of the model structure. Generally, there are five, six, or even 10 layers of hidden layer nodes; 2) this clearly highlights the importance of feature learning. That is to say, through layer-by-layer feature transformation, the Feature Representation of the sample in the original space is transformed into a new feature space, making classification or prediction easier. Compared with the artificial rule feature construction method, using big data to learn features is more capable of describing the rich internal information of data.

Deep Learning combines lower-layer features to form a more abstract high-level representation of attribute classes or features to discover distributed Feature Representation of data.

To overcome problems in neural network training, DL adopts a different training mechanism than neural network. In traditional neural networks, back propagation is used. In short, iterative algorithms are used to train the entire network, and initial values are randomly set to calculate the output of the current network, then, the parameters of the previous layers are changed based on the difference between the current output and the label until convergence (the whole is a gradient descent method ). Deep Learning is a layer-wise training mechanism. The reason for this is that if the back propagation mechanism is adopted, for a deep Network (more than 7 layers), the residual propagation to the front layer has become too small, the so-called gradient diffusion (gradient diffusion) occurs ).

Rb4.09.03 RBM