Learning algorithm of Ann training algorithm based on traditional neural network

Learning/Training Algorithm classification

The different types of neural networks correspond to different kinds of training/learning algorithms. Therefore, according to the classification of neural networks, the traditional neural network learning algorithms can be divided into the following three categories:

1 feedforward Neural Network Learning Algorithm-----(feedforward neural Network)

2 Feedback-based neural network learning algorithm------(Feedback neural network)

3 self-organizing neural network learning algorithm------(self-organizing neural network)

The following three kinds of typical neural network models are used to illustrate the differences and similarities of these three kinds of learning algorithms respectively.

Although there are three different kinds of training algorithms for different network models, these three kinds of training algorithms can be classified into two types of machine training methods, namely, supervised learning algorithm and unsupervised type

Learning algorithms. In the course of 20-30 years ' study of neural network learning algorithms, scientists have put forward and constructed different types of training algorithms by using supervised learning algorithm and unsupervised learning algorithm separately or in combination.

Its improved algorithm. Thus, it is concluded that today's neural network training algorithms can be categorized into supervised learning algorithm and unsupervised learning algorithm, which is also reflected in the DBNS network learning of deep learning in the following explanation.

To. Of course, a semi supervised learning method is also proposed, which is defined as.

Semi-supervised learning (semi-supervised Learning) is a key problem in the field of pattern recognition and machine learning, and it is a learning method combining supervised learning and unsupervised learning. It mainly considers how to use a small number of labeled samples and a lot of not labeled samples for training and classification problems. Semi-supervised learning is of great practical significance in reducing the cost of labeling and improving the performance of learning machines.

Semi-supervised learning is a combination of supervised learning algorithm and unsupervised learning algorithm, can be considered as the combination of two methods, the root cause is also attributed to two types of learning algorithms, and therefore also escape the supervision of learning and unsupervised learning domain circle, where we no longer in-depth discussion of the semi-supervised learning algorithm.

In the summary of the following traditional neural network training algorithms, we also specify the relationship between the specific training algorithm and the supervised learning algorithm and the unsupervised learning algorithm. BP neural Network Training algorithm following we analyze the BP neural network learning process. The basic steps of the learning algorithm can be summarized as follows:

1, initialize the network weights and neurons threshold (the simplest way is to randomly initialize)

2, forward propagation: According to the Formula One layer of the calculation of the hidden neurons and output layer of neurons in the input and output.

3. Back propagation: Correct weights and thresholds according to formula

Until the termination condition is met.

The algorithm can judge the forward propagation result by a certain function, and revise the network parameters through the post propagation process, so the traditional BP network training process could be summed up as a kind of typical supervised learning process.

BP is the English abbreviation for post propagation, so what is the object of communication. What is the purpose of communication. The way to spread is back, but what does that mean?

The object of propagation is error, the purpose of propagation is to get the estimation error of all layers, the latter is to say that the error of the front layer is deduced from the back layer error:

That is, BP's idea can be summed up to use the output error to estimate the direct leading layer of the output layer error, and then use this error to estimate the error of the previous layer, so that a layer of the reverse pass, it is all the other layers of error estimates. "BP neural network model topology structure includes input layer (inputs), hidden layer (hide layer) and output layer (outputs layer)"

Let's look at one of the simplest three-storey BP:

The BP network can learn and store a large number of input-output pattern mapping relationships without revealing the mathematical equations that describe the mapping relationship beforehand. ”

BP uses a function called activation to describe the relationship between layer and layer output, thus simulating the interaction of neurons in each layer.

Activation functions must satisfy all the conditions that can be guided everywhere. Then the more commonly used is an activation function called the S-type function:

So why is the above function called an S-type function:

Let's look at its form and its derivative form:

Derivative of p.s S-type function:

The purpose of neural network learning is to learn a model that can output one of our desired outputs to the input. The way of learning: Changing the connection weights of the network under the stimulation of the external input sample: Dynamic adjustment of the weights of each connection

The core of learning:

Weight adjustment rules, that is, in the learning process in the network of neurons in the connection of the change in the right to a certain adjustment rules.

Two, supervised BP model training process

1. The idea

Supervised BP model training shows that we have a training set that includes: input X and the output Y that it is expected to have

So for the current BP model, we can get the error of the needle for the training set.

So the core idea of BP is that the output error is somehow transmitted through the layers of the input layer through the hidden layer, and some form of this is actually:

That is, the process of "the forward propagation of signals----the reverse propagation of error > errors":

2. The specific

This explains that the weights are revised according to the partial derivative of the error to the weight value:

References

Http://www.360doc.com/content/13/1217/13/9282836_337854682.shtml