Deep learning helps AI go farther

To say deep study, you have to mention Geoffrey Hinton.


, Geoffrey Hinton.


Geoffrey Hinton is a leading figure in deep Learning (deep learning) and is one of the world's outstanding researchers in the field of machine learning and artificial intelligence.


Geoffrey Hinton is a Ph. D. In experimental psychology at the British http://www.aliyun.com/zixun/aggregation/13592.html > Cambridge University, and later received a PhD in artificial intelligence, He is also a founding director of the Gatsby Neuroscience Division at the University of London. He studied the world by using neural networks for learning, memory, perception, and symbolic processing, and has more than 200 publications in this field.


Geoffrey Hinton's leading research in these areas, which includes machine learning in cutting-edge areas of modern science, and how machines perform identification functions in large, complex data, has helped Google make more than one big step forward in neural network learning and voice.


II, Geoffrey Hinton and deep learning

A brief introduction to
depth study

The concept of
depth learning was first proposed by Geoffrey Hinton and others in 2006. It is a new field in machine learning, whose motive is to build and simulate the neural network of human brain, which imitates the mechanism of human brain to interpret data, such as image, sound and text. Deep learning is a kind of unsupervised learning.

The concept of
depth learning is derived from the study of artificial neural networks. Multilayer perceptron with multiple hidden layers is a deep learning structure. Depth learning forms a more abstract high-level representation of attribute classes or features by combining low-level features to discover distributed feature representations of data.


depth learning based on confidence degree network (DBN) puts forward unsupervised greedy hierarchical training algorithm, which brings hope to solve the problem of deep structure-related optimization, and then puts forward the deep structure of multilayer automatic encoder. In addition, the convolution neural network proposed by LeCun is the first real multilayer structure learning algorithm, which uses spatial relative relation to reduce the number of parameters to improve the training performance.


Basic Concept


Depth (Depth)


The computation involved in generating an output from an input can be represented by a flow graph: A flow diagram is a graph that can represent computations in which each node represents a basic calculation and a computed value (the computed result is applied to the value of the node's child node). Consider such a set of computations, which can be allowed in every node and possible graph structure, and define a function family. The input node has no children, the output node has no father.


A special attribute of this flow diagram is the depth (depth): the length of the longest path from one input to one output.


traditional Feedforward neural networks can be considered to have a depth equal to the number of layers (for example, for the output layer with a hidden layer plus 1). The SVMS has a depth of 2 (one corresponds to the kernel output or feature space, and the other corresponds to the linear blending of the resulting output).


need to use in-depth learning to solve the problem has the following characteristics:

There is a problem with insufficient depth of
.

The
brain has a deep structure.


cognitive processes are layered and abstracted gradually.


depth will cause problems


in many cases, depth 2 is sufficient to represent any function with a given target precision. But the price is that the number of nodes (such as the number of calculations and parameters) required in the diagram can become very large. Theoretical results confirm that the number of nodes that are in fact required to grow with the input size exponent is present.


we can consider the depth architecture as a factor decomposition. Most of the randomly selected functions cannot be effectively represented, either in deep or shallow architectures. But many that can be effectively represented by the depth architecture cannot be expressed efficiently in shallow architectures. The existence of a tight and deep representation means that there is a structure in the potential function that can be represented. If there is no structure, it will not be well generalized.


Brain has a deep architecture


For example, the visual cortex is well studied and shows a series of regions, each of which contains an input representation and a stream of signals from one to the other (this ignores the association at some levels of parallel paths and is therefore more complex). Each layer of this feature level represents input on a different abstraction layer and has more abstract features at the upper level of the hierarchy, which they define according to low-level features.


Note that the expression in the brain is tightly distributed and pure in the middle: they are sparse: 1% of the neurons are active at the same time. Given a large number of neurons, there is still a very efficient (high level) representation.


cognitive processes are layered and gradually abstracted


the idea and concept of human hierarchy;


Humans first learn simple concepts and then use them to express more abstract


engineers decompose tasks into multiple levels of abstraction;


Learn/Discover these concepts (knowledge fail with no introspection?) ) is very good. Introspection of the expressive concept of language also suggests a sparse representation: only a small part of all possible words/concepts can be applied to a particular input (a visual scene). [1] [3]

The core thought of
deep learning


The learning structure as a network, the core idea of deep learning is as follows:


① unsupervised learning pre-train for each layer of network;


② each time using unsupervised learning to train only one layer, its training results as the input of its senior level;


③ with supervised learning to adjust all layers


the successful application of deep learning


1, speech recognition


Microsoft researchers, in collaboration with Hintion, first introduced RBM and DBN into speech recognition acoustic model training and achieved great success in large vocabulary speech recognition systems, which reduced the error rate of speech recognition by 30%.


2, smartphone voice search


2012, Google's Android operating system of speech recognition by leaps and bounds, precisely because of the relationship between deep learning. Because the depth learning neural network allows more accurate speech training, so the success rate of speech recognition is greatly improved, especially in noisy environment, voice search results have been improved. Overnight, the error rate of smartphone speech recognition system dropped to 25%, which makes many commentators think that the voice search for Android chickens is smarter than Apple's Siri.


3, image recognition


Last June, Google demonstrated the largest neural network ever, with more than 1 billion nodes in the network, and successfully extracted 10 million images of cat images from YouTube videos to classify YouTube videos to 16% accuracy. This number, though seemingly small, has increased by 70% over the previous generation. It is to be noted that YouTube's system uses deep learning to divide video into 22,000 categories, many of which are indistinguishable from ordinary people. When the classification was reduced to 1000, the accuracy rate of system recognition increased to 50%. Without deep learning, Google's neural network is less powerful.


depth study is helping AI go farther!