BP Neural network

BP (back propagation) neural network was proposed by the team of scientists led by Rumelhart and McCelland in 1986, which is one of the most widely used neural network models, which is a multilayer Feedforward network trained by error inverse propagation algorithm. The BP network can learn and store a large number of input-output pattern mapping relationships without having to reveal the mathematical equations that describe the mapping relationship beforehand. Its learning rule is to use the steepest descent method, through the reverse propagation to continuously adjust the network weights and thresholds, so that the network error squared and minimum. The BP neural network model topology includes the input layer (inputs), the hidden layer (hidden layers), and the output layer.

Let us now analyze these words:

• "is a multilayer Feedforward network trained by error inverse propagation algorithm"

BP is the English abbreviation for the post-propagation, so what is the object of transmission? What is the purpose of communication? The way to spread is back, but what does that mean?

The object of propagation is error, the purpose of propagation is to get the estimation error of all layers, and the latter is to deduce the front layer error by the back layer error:

That is, the idea of BP can be summed up

Using the error of output to estimate the error of the direct leading layer of the output layer, and then using this error to estimate the error of the previous layer, the error of all the other layers is obtained when the inverse of the layer is passed down.

• "The BP neural Network model topology consists of input layers (inputs), hidden layers (hide layer), and output layer"

Let's look at one of the simplest three-layer BP:

• "The BP network can learn and store a large number of input-output pattern mapping relationships without having to reveal the mathematical equations that describe this mapping relationship beforehand." ”

BP uses a function called activation to describe the relationship between layer and layer output, thus simulating the interaction between the neurons in each layer.

The activation function must satisfy conditions that are everywhere. Then the more commonly used is an activation function called the S-type function:

So why is the above function called an S-type function?

Let's look at its shape and the shape of its derivative:

P.S. Derivative of the S-type function:

Neural Networks for learning purposes:

I want to be able to learn a model that can output a desired output to the input. The way of learning: constantly changing the connection weights of the network under the stimulation of the external input sample the essence of Learning: Dynamic adjustment of the value of each connection weight

The core of learning:

Weight adjustment rules, that is, in the learning process, the connection rights of each neuron in the network changes based on a certain adjustment rules.

Second, the supervised BP model training process

1. Thought

Supervised BP model training indicates that we have a training set that includes: input X and the output Y that it is expected to have

So for a current BP model, we can get the error of its needle for the training set.

So the core idea of BP is: The output error in some form through the hidden layer to the input layer of the anti-pass, some of the form here is actually:

It is a process of "forward propagation of signals----the reverse propagation of > errors":

2. Specific

This explains that the weights are revised according to the partial derivative of the error to the weighted value:

(Refer to: http://www.cnblogs.com/wengzilin/archive/2013/04/24/3041019.html or http://blog.csdn.net/byxdaz/article/ details/534855 is a good sum.)

BP Neural network