[Pattern recognition] multi-layer sensor MLP

As we can see in the previous section, a single sensor is the simplest machine that can be "learned" to solve linear data classification problems. But he cannot solve the non-linear problem. For exampleXOR Problems: () (-1,-1) belongs to the same class, while (1,-1) (-) belongs to the second category, and cannot be correctly classified by a single sensor.

That is, the sensor, analyzed in Minsky and Papert's monograph "sensor", can only solve the so-called first-level predicate logic problems: And, or (OR, however, it cannot solve the issue of multi-level predicates such as variance or (XOR.

Implement non-linearity with multiple sensors

Although a single sensor cannot solve an exception or problem, you can combine multiple sensors to split Complex Spaces. For example:

The two layers of sensor are combined according to a certain structure and coefficient. The first layer of sensor implements two linear classifiers, separates the feature space, and adds a layer of sensor to the output of the two sensors, the exclusive or operation can be implemented.

That is, it is combined by multiple sensors:

Nonlinear classification plane, where θ (·) represents a step function or symbol function.

Multi-layer sensor Neural Network

In fact, the above model isMulti-layer sensor Neural Network (multi-layer perceptron neural networks, MLP neural netwoks). Each node in the neural network is a sensor. The basic function of neurons in the model bio-neural network is that electrical signals from the outside (environment or other cells) are transmitted to neurons through the syn, when the total number of signals received by a cell exceeds a certain threshold, the cell is activated, and an electrical signal is sent to the next cell through the axon to complete external information processing.

However, sensor LearningAlgorithmIt cannot be applied directly to parameter learning of a multi-layer sensor model. Therefore, the original learning scheme was as follows: aside from the last neuron, the weights of all other neurons are fixed in advance. The learning process is just to use the sensor learning algorithm to learn the weights of the last neuron. In fact, this is equivalent to converting the original feature space to a new feature space through the first layer of neurons. Each neuron in the first layer forms one dimension of the new space, then, a linear classifier is constructed using the sensor Learning Algorithm in the new feature space. Obviously, because the neuron weights at the first layer need to be manually specified, the performance of the model depends largely on whether an appropriate first-level neuron model can be designed, this depends on the knowledge of the problems and data we are facing, and there is no way to solve the first-level neuron parameters for any problem.

(Reprinted please indicate the author and Source: http://blog.csdn.net/xiaowei_cqu is not allowed for commercial use)