Linear neural network model and learning algorithm __ Neural network

The linear neural network is similar to the perceptron, but the activation function of the linear neural network is linear rather than the hard transfer function, so the output of the linear neural network can be any value, and the output of the perceptron is not 0 or 1. Linear neural networks, like Perceptron, can only solve linear and fractal problems. Therefore, the linear neural network has the same limitations as the perceptron. linear neural Network model

Linear neurons have similar structures to perceptron neurons, and the only difference is that linear neurons use linear transfer function Purelin, so unlike perceptron neural networks, the output of a linear neural network can be arbitrary.

The output of a linear neuron can be computed by the following formula Y=purelin (v) =purelin (ω→⋅p→+b) =ω→⋅p→+b y = Purelin (v) = Purelin (\overrightarrow \omega \cdot \ Overrightarrow p + b) = \overrightarrow \omega \cdot \overrightarrow p + b

When the output y equals 0, you can draw their boundaries. The input vector located above the dividing line can produce a network output greater than 0, and the input vector below the dividing line can produce a network output less than 0. So the linear neuron can only approximate a linear function, but not the approximation of the nonlinear function. Its limitation is the same as that of perceptual neural network. Learning algorithm of linear neural network

The learning rule adopted by linear neural network is widrow-hoff learning rule, also known as minimum mean error (LMS) learning algorithm, which reduces the training error of network based on the principle of negative gradient descent. The least mean square error learning algorithm also belongs to the supervised class learning algorithm.

The

assumes that pk= (P1,P2,⋯,PR (k)) P_k = (p_1, p_2, \cdots, P_r (k)) represents the input vector of the network, dk= (D1 (k), D2 (k), ⋯,ds (k)) D_k = (D_1 (k), D_2 (k), \CDO TS, d_s (k)) represent the desired output vector of the network, yk= (Y1 (k), Y2 (k), ⋯,ys (k)) Y_k = (Y_1 (k), Y_2 (k), \cdots, y_s (k)) represents the actual output vector of the network, where k=1,2,⋯,m k = 1, 2, \cdots, M represents the number of pairs of input vectors and corresponding expected output vector samples. The rule of the LMS system is to reduce the mean value of these error squares, as defined below: Mse=1m∑k=1me2 (k) =1m∑k=1m (d (k) −y (