Summary of Ann Training algorithm based on traditional neural network

Summary of Ann Training algorithm based on traditional neural network

Learning/Training Algorithm classification

The different types of neural networks correspond to different kinds of training/learning algorithms. Therefore, according to the classification of neural networks, the traditional neural network learning algorithms can be divided into the following three categories:

1) Feed-forward neural network Learning Algorithm-----(feedforward neural Network)

2) Feedback Neural Network Learning algorithm------(feedback type neural network)

3) Self-organizing neural network learning algorithm------(self-organizing neural network)

Below, we will expound the differences and similarities of these three kinds of learning algorithms by three kinds of typical neural network models.

Although there are three different types of training algorithms for different network models, the three kinds of training algorithms can be attributed to two types of machine training methods, namely supervised learning algorithm and unsupervised type

Learning algorithms. In the 20-30 years of Neural network learning algorithm research, scientists often use supervised learning algorithms and unsupervised learning algorithms to separate or mix, put forward and construct different types of training algorithms and

Its improved algorithm. So it concludes that nowadays neural network training algorithms can be classified into supervised learning algorithms and unsupervised learning algorithms, which will also be reflected in the DBNS network learning in the follow-up deep learning.

To. Of course, a semi-supervised learning method is also proposed, which is defined as.

Semi-supervised learning (semi-supervised learning) is a key problem in the field of pattern recognition and machine learning, and it is a learning method combining supervised learning with unsupervised learning. It mainly considers how to use a small number of labeling samples and a lot of unlabeled samples for training and classification problems. Semi-supervised learning is of great practical significance for reducing labeling cost and improving learning machine performance.

Semi-supervised learning is a combination of supervised learning algorithm and unsupervised learning algorithm, which can be considered as the combination of two methods, the root of which is also attributed to two kinds of essential learning algorithms, so it is also not able to escape the circle of supervised learning and unsupervised learning, here we will not further discuss the semi-supervised learning algorithm.

In the following summary of the traditional neural network training algorithm, we will also specify the specific training algorithm and supervised learning algorithm and unsupervised learning algorithm relationship.

BP Neural Network Training algorithm

Below we analyze the BP neural network learning process. The basic steps of the learning algorithm can be summarized as follows:

1, initialize the network weights and neurons threshold (the simplest way is to initialize randomly)

2. Forward propagation: The input and output of the hidden-layer neurons and the output-layer neurons are calculated in the first layer of the formula.

3. Back propagation: Correct weights and thresholds according to the formula

Until the termination condition is met.

The algorithm can judge the forward propagation result by certain function, and revise the network parameters through the back propagation process, and play the role of supervised learning, so the traditional BP network training process may be summed up as a class of typical supervised learning process.

BP is the English abbreviation for the post-propagation, so what is the object of transmission? What is the purpose of communication? The way to spread is back, but what does that mean?

The object of propagation is error, the purpose of propagation is to get the estimation error of all layers, and the latter is to deduce the front layer error by the back layer error:

That is, the idea of BP can be summed up

Using the error of output to estimate the error of the direct leading layer of the output layer, and then using this error to estimate the error of the previous layer, the error of all the other layers is obtained when the inverse of the layer is passed down.

• "The BP neural Network model topology consists of input layers (inputs), hidden layers (hide layer), and output layer"

Let's look at one of the simplest three-layer BP:

• "The BP network can learn and store a large number of input-output pattern mapping relationships without having to reveal the mathematical equations that describe this mapping relationship beforehand." ”

BP uses a function called activation to describe the relationship between layer and layer output, thus simulating the interaction between the neurons in each layer.

The activation function must satisfy conditions that are everywhere. Then the more commonly used is an activation function called the S-type function:

So why is the above function called an S-type function?

Let's look at its shape and the shape of its derivative:

P.S. Derivative of the S-type function:

Neural Networks for learning purposes:

I want to be able to learn a model that can output a desired output to the input. The way of learning: constantly changing the connection weights of the network under the stimulation of the external input sample the essence of Learning: Dynamic adjustment of the value of each connection weight

The core of learning:

Weight adjustment rules, that is, in the learning process, the connection rights of each neuron in the network changes based on a certain adjustment rules.

Second, the supervised BP model training process

1. Thought

Supervised BP model training indicates that we have a training set that includes: input X and the output Y that it is expected to have

So for a current BP model, we can get the error of its needle for the training set.

So the core idea of BP is: The output error in some form through the hidden layer to the input layer of the anti-pass, some of the form here is actually:

It is a process of "forward propagation of signals----the reverse propagation of > errors":

2. Specific

This explains that the weights are revised according to the partial derivative of the error to the weighted value:

References

Http://www.360doc.com/content/13/1217/13/9282836_337854682.shtml

Summary of Ann Training algorithm based on traditional neural network