Deep learning and neural Network

The article was transferred from the deep learning public number

Deep learning is a new field in machine learning that is motivated by the establishment and simulation of a neural network for analytical learning of the human brain, which mimics the mechanisms of the human brain to interpret data, examples, sounds and texts. Deep learning is a kind of unsupervised learning.

The concept of deep learning derives from the research of artificial neural networks. Multilayer perceptron with multiple hidden layers is a kind of deep learning structure. Deep Learning represents attribute categories or characteristics by combining lower-level features to form more abstract higher levels, to discover distributed feature representations of data.

Deep learning itself is a machine learning branch, simple can be understood as the development of neural network. About twenty or thirty years ago, the neural network was once a particularly fiery direction in the ML field, but it was slowly fading out for several reasons, including the following:

1) relatively easy to fit, the parameters are difficult to tune, and need a lot of trick;

2) Training speed is relatively slow, at a lower level (less than or equal to 3) the effect is not better than other methods;

So in the middle there are about more than 20 years, the neural network is concerned about very little, this period of time is basically SVM and boosting algorithm of the world. However, a foolish old gentleman Hinton, he insisted on down, and eventually (and others together Bengio, Yann.lecun, etc.) commission a practical deep learning framework.

There are many differences between deep learning and traditional neural networks.

The same is the deep learning using a similar hierarchical structure of neural network, the system consists of input layer, hidden layer (multilayer), the output layer composed of multi-layer network, only the adjacent layer nodes are connected, the same layer and the cross-layer nodes are not connected to each other, each layer can be regarded as a logistic regression model; This hierarchical structure is relatively close to the structure of the human brain.

In order to overcome the problems in neural network training, DL adopts the training mechanism which is very different from the neural network. Traditional neural network, the use of the back propagation way to do, the simple is to use an iterative algorithm to train the entire network, randomly set the initial value, calculate the current network output, and then according to the difference between the current output and label to change the parameters of the previous layers, Until convergence (the whole is a gradient descent method). And deep learning is a layer-wise training mechanism on the whole. The reason for this is because, if you use the back propagation mechanism, for a deep network (above 7 layers), the residual spread to the front of the layer has become too small, the emergence of so-called gradient diffusion (gradient diffusion).

Deep Learning vs Neural networks

The concept of deep learning derives from the research of artificial neural networks. Multilayer perceptron with multiple hidden layers is a kind of deep learning structure. Deep Learning represents attribute categories or characteristics by combining lower-level features to form more abstract higher levels, to discover distributed feature representations of data.
The concept of deep learning was presented by Hinton and others in 2006. Based on the belief degree network (DBN), a non-supervised greedy layer training algorithm is proposed, which brings hope for solving the problem of deep structure-related optimization, and then proposes the deep structure of multilayer automatic encoder. In addition, the convolution neural network proposed by LeCun is the first real multi-layer structure learning algorithm, which uses spatial relative relationships to reduce the number of parameters to improve training performance.

Deep learning and neural Network