BP algorithm based on multilayer neural network

Principles of training multi-layer neural network using backpropagation

The project describes teaching process of multi-layer neural network employing backpropagation algorithm. To illustrate this process, the three layer neural network with the inputs and one Output,which are shown in the picture Bel OW, is used: Each neuron is composed of units. First unit adds products of weights coefficients and input signals. The second unit realise nonlinear function, called neuron activation function. Signaleis adder output signal, andy = f (e)is output signal of nonlinear element. Signalyis also output signal of neuron. To teach the neural network we need training data set. The training data set consists of input signals (X1andX2) assigned with corresponding target (desired output)Z. The network training is an iterative process. In each iteration weights coefficients of nodes is modified using new data from training data set. Modification is calculated using algorithm described Below:each teaching step starts with forcing both input signals from Training set. After this stage, we can determine output signals values for each neuron in each network layer. Pictures below illustrate how signal is propagating through the network, SymbolsW (XM) nRepresent weights of connections between network inputXMand neuronNIn input layer. SymbolsynRepresents output signal of neuronN. Propagation of signals through the hidden layer. SymbolswmnRepresent weights of connections between output of neuronmand input of neuronNIn the next layer. Propagation of signals through the output layer. In the next algorithm step the output signal of the networkyis compared with the desired output value (the target), which was found in training data set. The difference is called error signalDof output layer neuron. It is impossible to compute error signal for internal neurons directly, because output values of these neurons are unknown . For many years the effective method for training multiplayer networks have been unknown. Only the middle Eighties the backpropagation algorithm have been worked out. The idea was to propagate error signalD(Computed in a teaching step) back to all neurons, which output signals were input for discussed neuron. The weights ' coefficientswmnUsed to propagate errors back is equal to this used during computing output value. The direction of data flow is changed (signals was propagated from output to inputs one after the other). This technique are used for all network layers. If propagated errors came from few neurons they is added. The illustration is below: When the error signal for each neuron are computed, the weights coefficients of each neuron input node may modified. In formulas belowDF (e)/deRepresents derivative of neuron activation function (which weights is modified). CoefficienthAffects network teaching speed. There is a few techniques to select this parameter. The first method is to start teaching process with large value of the parameter. While weights coefficients was being established the parameter is being decreased gradually. The second, more complicated, method starts teaching with small parameter value. During the teaching process the parameter is being increased when the teaching are advanced and then decreased again in the Final stage. Starting teaching process with low parameter value enables to determine weights coefficients signs. References Ryszard Tadeusiewcz "Sieci Neuronowe", Kraków 1992
Mariusz Bernackiprzemys?aw W?odarczykMgr in?. Adam Go?da (2005) Katedra Elektroniki AGH
Last modified:06.09.2004 Webmaster

BP algorithm based on multilayer neural network