The principle of RBF neural networks has been introduced in my blog, "RBF Neural Network for machine learning", which is not repeated here. Today is to introduce the common RBF neural Network learning Algorithm and RBF neural network and multilayer perceptron network comparison.
RBF Neural Network Learning Algorithm
The generalized RBF neural network structure is as follows:
The N-M-L structure corresponds to an n-dimensional input, and M data center points centers,l outputs.
Common learning algorithms for RBF networks
The design of RBF network includes structural design and parameter design. The structure design mainly solves the problem of how to determine the number of hidden nodes in the network. Parameter design generally need to consider including 3 parameters: the data Center and extension constants of each base function, and the weights of the output nodes . When the full RBF network structure is used, the number of hidden nodes is the sample number, the data center of the base function is the sample itself, and the parameter design only needs to consider the extension constants and the weights of the output nodes. When using the generalized RBF network structure, the learning algorithms of RBF Network include: How to determine the number of hidden nodes in the network, how to determine the data center and extension constants of each radial basis function, and how to correct the output weights.
According to the data Center value method, the RBF network design method can be divided into two kinds.
First Class method: The data center is selected from the sample input. In general, sample-dense local center points can be more appropriate, the sample sparse local center point can be less, if the data itself is evenly distributed, the center point can also be evenly distributed. In short, the selected data centre should be representative. The extended constants of radial basis functions are based on the data center
Scattered and determined, in order to avoid each radial basis function is too sharp or Taiping, one option is to set all radial basis functions extension constant to: Max (d)/sqrt (2M), M is the data center point,Max (d) is the maximum distance between the selected data center.
the second kind of method: the self-organization choice of data center. A variety of dynamic clustering algorithms are used to select the data center, and the location of the data center should be adjusted dynamically during the learning process. The common method is K-means clustering, which has the advantage of determining the extension constants of each hidden node according to the distance between each cluster center. Because the number of hidden nodes of RBF network has great influence on its generalization ability, it is the first problem to find a reasonable method to determine the number of clusters, which is to be solved in the design of RBF network by clustering method. In addition to clustering algorithm, there are gradient training methods resource allocation network ran and so on.
1. clustering algorithms for data centers
Using K-means clustering algorithm, the M data center point and point expansion constants are obtained. using K-means Clustering algorithm, the M data center point and point expansion constants are obtained.Using K-mean Clustering algorithm to obtain the center and extension constants of each radial basis function, the second step of the blended learning process is to use supervised learning algorithm to obtain the weight of the output layer, often using the least mean square algorithm (LMS), the input vector of the algorithm is the output vector of the hidden node. A more straightforward approach is to use pseudo-inverse method to calculate directly. Set
When input is X p, the output of the J hidden Node is ΦPJ =φ (‖xp-cj‖), p = 1, 2, ..., p, j = 1, 2, ...,
M, the hidden layer output matrix is φ1 = (ΦPJ) pxm If the pending output weights of the RBF network are w = (W1, w2, ..., W M), then the network output vector is F (X) =φ1w
To make the network output vector equal to the teacher signal D, the W can be obtained by pseudo-inverse φ+ of φ1:
2. monitoring learning algorithms for data centers
In the most general case, the center of the hidden node RBF function, the extended constant and the output layer weights are trained by the supervised learning algorithm, that is, all parameters undergo an error correction learning process, and the method uses the BP algorithm to train the multi-layer perceptron principle similar. The following is an example of a single output RBF network, which introduces a gradient descent algorithm.
Define the target function as:
in the formula P is the number of training samples, EI is the error signal when I sample input, defined as
The threshold value is ignored in the output function of the above formula.
To minimize the objective function, the correction amount of each parameter should be proportional to its negative gradient, i.e.
The above objective function is the sum of the errors caused by all training samples, and the derived parameter correction formula is a batch-type adjustment, that is, all sample input is adjusted once after one round. The objective function can also be defined as the instantaneous value form, which is the error caused by the current input sample: E = 0.5e*e. The parameter of minimizing the objective function in the upper form is formally a single-sample training mode, i.e.
3. Implementation of supervised learning algorithm based on data center See the code below for details:
Comparison of RBF neural network and multilayer Perceptron networkRBF Network and multilayer perceptron are nonlinear multilayer forward networks, both of which are general-purpose approximations. For any
multilayer perceptron, there is always a RBF network that can replace it, and vice versa. However, there are also
many differences between the two networks. The
①RBF Network has only one hidden layer, and the hidden layer of a multilayer perceptron can be a layer or multilayer. The neuron model of the
② multilayer perceptron is the same as the hidden layer and output layer. But the hidden layer neuron and the output layer neuron of RBF network not only have different models, but also play an important role in the network.
③RBF The hidden layer of the network is nonlinear and the output layer is linear. However, when using multilayer perceptron to solve the problem of pattern classification, its hidden layer and output layer are usually selected as non-linear. When solving nonlinear regression problems with multilayer perceptron, a linear output layer is usually chosen. The base function of the
④RBF network calculates the Euclidean distance between the input vector and the center, while the excitation function of the hidden unit of the multilayer Perceptron calculates the inner product between the input unit and the connection weight value. The
⑤RBF network uses a nonlinear function of local exponential decay, such as a Gaussian function, to approximate a nonlinear input-output map. The implicit nodes of multilayer perceptron (including BP net) adopt the input mode and the inner product of weight vector as the independent variable of activation function, while the activation function uses Sigmoid function or hard limit amplitude function, so multilayer perceptron is global approximation to nonlinear mapping. The most notable feature of RBF network is that the implicit node takes the distance of the input mode and the center vector (such as Euclidean distance) as the function's independent variable, and uses the radial basis function (such as the Gaussian function) as the activation function. Radial basis functions a central point of the N-dimensional space has radial symmetry, and the farther the neuron's input is from the center point, the less the neuron activates. This feature of hidden nodes is often referred to as "local characteristics".
RBF network has a wide application because it can approximate arbitrary nonlinear functions, and is able to deal with the inherent difficult regularity of the system and has fast learning convergence speed. At present, RBF Network has been successfully used in nonlinear function approximation, time series analysis, data classification, pattern recognition, information processing, image processing, System modeling, control and fault diagnosis.
***********************2015-8-7
Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.
RBF Neural Network Learning algorithm and its comparison with multilayer Perceptron
