This video is fun to talk about the development of deep learning

In view of my knowledge of machine learning and statistics, insufficient, temporarily do not translate. I just write down the original English, there may be mistakes, quite fun. Also saw a piece of Chinese article, found in the video recorded in the Deep learning development of the time point.

First, record the difference between deep learning and neural networks. Deep learning uses many hidden layer neural networks to learn, and for training speed, deep learning chooses the method of training by layers.

First wave: Shallow learning

The 80 's back propagation invented, which brings new hope to machine learning. At this time the artificial neural network, also known as multilayer Perceptron, actually contains only a shallow layer model of the hidden layer node.

In the 90 's, a variety of shallow machine learning models were presented, such as support vector machines (svm,support vector machines), boosting, maximum entropy method (e.g. LR, Logistic Regression), etc. The structure of these models can basically be seen with a layer of hidden nodes (such as SVM, boosting), or no hidden layer nodes (such as LR). These models have achieved great success in both theoretical analysis and application. Compared with the theoretical analysis of the difficulty, coupled with training methods require a lot of experience and skills, so this period of shallow artificial neural network rather relatively quiet.

Second wave: Deep learning

In 2006,--geoffrey Hinton, a professor at the University of Toronto in Canada, and his student Ruslan Salakhutdinov, published an article in the science of top academic journals, which opened the wave of deep learning in academia and industry. This post has two main messages: 1. Many hidden layers of artificial neural networks have excellent characteristics of learning ability, the characteristics of learning to have a more essential characterization of data, which is conducive to visualization or classification; 2. The difficulty of the deep neural network in training can be effectively overcome by the "layer initialization" (Layer-wise pre-training), in this article, the level-by-layer initialization is realized by unsupervised learning.

In fact, simple translation is the time period for the development of deep learning.

1983.

---One researcher

Deep belive nets, they is more efficient than kernels, I am going to use them again and again.

Jeff Hinton is the leader, follow the leader. But what is the origins of this incredible successful story.


1983.

---Girl
Hello Professor Hinton, please tell us about your.

---Hinton
Something incredable happened, I know how the brain works. Wow, it's about mans machine, but how the brain efficiently estimates the gradient of the partion function.

---Girl
Did your made progress with the man made machine.

---Hinton
I have found something better, the back propagation algorithm, the brains must deploy this tricks to deploy this tricks to Compute gradients.

---Girl
Hello Professor Hinton, did you confirm that the brain does back propagation.

---Hinton
I decide not to answer the question.

1993.

---Girl
Hello Professor Hinton, did your confirm the brain does back propagation.

---Hinton
I try not to answer the question. But I can-finally tell the brain works. It is grapical model, it's trained thanks to variation to Proximation.


2000.

---Hinton
I know how the brain works.

---Girl
Of course it is variational.


Not at all it uses contrasted dirvagence to approximates the gradients of the partition function.


2006.

---Hinton
This time I got it, deep learning.

---Girl
I guess that's how the brain works?

---Hinton
Yes, but I am supervised by pretty training about multiple layers.


2010.

---Girl
I don ' t think can discover how the brain works.

---Hinton
And I never think that Michael Jordan could play basketball alone. I tell you I am 100% sure then the brain is using GA matrix transformation.
What ever, is you supposed isn't to organize deep learning workshop.

---Hinton
Is you crazy? They is going to make me-stupid jokes at the closing Backrid,

---Girl
I See, and how does you envision the future about deep learnning?

---Hinton
Live long and prosper.

Deep learning, the dream of advancing AI: http://www.csdn.net/article/2013-05-29/2815479

Deep learning video:http://www.youtube.com/watch?v=mlxzufek-2e