Understanding Deep Learning

1. Current situation:

Deep learning is now very hot, and all kinds of meetings have to be stained with this point. Baidu Brain, Google's brain plan to engage in this. In some areas have achieved very good results, chip recognition, speech recognition, in the security field and even the identification of encryption protocols. The accuracy of the lab in the field of speech is over 90%.

2. The essence of deep learning

A typical machine learning sample, for example, is to obtain data from the beginning via a sensor (e.g. CMOS). Then through preprocessing, feature extraction, feature selection, and inference, prediction or recognition. The last part, the machine learning part, most of the work is done in this area, there are a lot of paper and research.

In the middle of the three parts, summed up is the characteristics of expression. Good feature expression plays a key role in the accuracy of the final algorithm, and the main calculation and testing work of the system is consumed in this part. However, this piece of practice is generally done manually. By manual extraction of features.

However, the manual selection of features is a very laborious, heuristic (requires expertise) approach, can be selected to a great extent by experience and luck, and its adjustment takes a lot of time. Since the manual selection of features is not very good, then can you automatically learn some features?                               The answer is YES! Deep learning is used to do this thing, see it's an alias Unsupervisedfeature learning, it can be as the name implies, unsupervised means not to participate in the selection process of character.

The essence of deep learning is to learn more useful features by building machine learning models with many hidden layers and massive training data, which ultimately improves the accuracy of classification or prediction. Therefore, the "depth model" is the means by which "characteristic learning" is the purpose. Different from the traditional shallow learning, the difference of deep learning is that: 1) emphasizes the depth of the model structure, usually has 5 layers, 6 layers, or even 10 layers of hidden layer nodes; 2) clearly highlights the importance of feature learning, that is to say, by changing the characteristics of the original space to a new feature space, This makes it easier to classify or predict. Compared with the method of constructing characteristics of artificial rules, the use of big data to learn the characteristics, more able to depict the rich intrinsic information of the data.

3. The relationship between deep learning and traditional neural networks

Deep learning itself is a machine learning branch, simple can be understood as the development of neural network. About twenty or thirty years ago, the neural network was once a particularly fiery direction in the ML field, but it was slowly fading out for several reasons, including the following:

1) relatively easy to fit, the parameters are difficult to tune, and need a lot of trick;

2) Training speed is relatively slow, at a lower level (less than or equal to 3) the effect is not better than other methods;

So in the middle there are about more than 20 years, the neural network is concerned about very little, this period of time is basically SVM and boosting algorithm of the world. However, a foolish old gentleman Hinton, he insisted on down, and eventually (and others together Bengio, Yann.lecun, etc.) commission a practical deep learning framework.

There are many differences between deep learning and traditional neural networks.

The same is the deep learning using a similar hierarchical structure of neural network, the system consists of input layer, hidden layer (multilayer), the output layer composed of multi-layer network, only the adjacent layer nodes are connected, the same layer and the cross-layer nodes are not connected to each other, each layer can be regarded as a logistic regression model; This hierarchical structure is relatively close to the structure of the human brain.

In order to overcome the problems in neural network training, DL adopts the training mechanism which is very different from the neural network. Traditional neural network, the use of the back propagation way to do, the simple is to use an iterative algorithm to train the entire network, randomly set the initial value, calculate the current network output, and then according to the difference between the current output and label to change the parameters of the previous layers, Until convergence (the whole is a gradient descent method). And deep learning is a layer-wise training mechanism on the whole. The reason for this is because, if you use the back propagation mechanism, for a deep network (above 7 layers), the residual spread to the front of the layer has become too small, the emergence of so-called gradient diffusion (gradient diffusion).

5. Summary

Deep learning is a multi-layered (complex) expression algorithm that automatically learns the potential (implicit) distribution of data to be modeled. In other words, the deep learning algorithm automatically extracts the classification needs of low-level or high-order features. High-level characteristics, one is that the feature can be graded (hierarchical) dependent on other characteristics, such as: For machine vision, the deep learning algorithm from the original image to learn to get its low levels of expression, such as edge detector, wavelet filter, and then on the basis of these low-level expression, and then establish the expression, For example, these low-level expressions of linear or nonlinear combinations, and then repeat the process, and finally get a higher expression.

Deep learning is able to get a better representation of the feature of the data, and because of the level of the model, many parameters, capacity enough, so the model has the ability to represent large-scale data, so for the image, This feature is not obvious (requires manual design and a lot of non-intuitive physical meaning) problem, can be in large-scale training data to achieve better results. In addition, from the perspective of Pattern recognition features and classifiers, the Deep learning Framework combines feature and classifiers into a framework that uses data to learn feature, In the use of the manual design feature to reduce the huge amount of work (this is the current industry engineers to pay the most effort), therefore, not only the effect can be better, but also has a lot of convenience, so it is a very noteworthy set of framework, every ML should be concerned about understanding.

Of course, the deep learning itself is not perfect, nor is it a weapon to solve any ml problem in the world, and should not be magnified to an omnipotent degree.

Understanding Deep Learning