Neural network (b) Curve fitting

Calculate Smart Jobs two

title : Optional Nonlinear classification or curve fitting problem, training and learning with BP network.
Optional topics:
The data in the following list is the 20-year road traffic volume data for a region, where the attributes "population", "number of vehicles" and "Road area" as input, attribute "road passenger volume" and "road freight" as output. Please fit this multi-input multi-output curve with a neural network.

(1) Neural network principle
The basic principle of the BP Network model processing information is: the input signal XI through the middle node (hidden layer Point) action on the output node, after the non-linear transformation, produce output signal YK, each sample of network training includes input vector x and expected output T, network output value Y and expected output value of T, the deviation between By adjusting the connection strength between the input node and the hidden layer node and the connection strength TJK and threshold of the wij and the hidden layer nodes, the error decreases along the gradient direction, and after repeated learning and training, the network parameters (weights and thresholds) corresponding to the minimum error are determined, and the training is stopped. In this case, the trained neural network can process the input information of similar sample, and handle the non-linear conversion information with the least output error.

(2) structure design of neural network
A three-layer neural network is used to implement the correlation, the input layer is 3 input, the hidden layer sets 8 nodes, and the output layer is set to 2 outputs.



Activation function:



Weight update:



The weight update steps for the neural network are:
(1) Initialize the connection weights of each layer, and set the learning rate and inertia coefficient.
(2) Enter a sample pair and compute the output values for each layer node.
(3) According to the gradient descent strategy, the parameter is adjusted by the negative gradient direction of the objective function.

The standard BP algorithm updates the connection right for only one training sample at a time, the parameters are updated very frequently, and the effect of updating different training samples may be "offset". Therefore, in order to finally achieve the least error in all training data, after reading the entire training set, and then updating the parameters, the training speed can be greatly accelerated. So in this problem, we use cumulative error to update weights.
Program code:

#-*-Coding:utf-8-*-"" "Created on Wed Oct (09:58:50) This file was used to fit the curve of traffic. @author: Hansyang "" "Import NumPy as NP import Matplotlib.pyplot as Plt def logsig (x): Return 1/(1+np.exp (-X)) #Orig Inal Data #Input: The pupulation, number of vehicle, Roadarea from 1990-2009 population=[20.55,22.44,25.37,27.13,29.45,30 .10,30.96,34.06,36.42,38.09,39.13,39.99,41.93,44.59,47.30,52.89,55.73,56.76,59.17,60.63] Vehicle=[ 0.6,0.75,0.85,0.9,1.05,1.35,1.45,1.6,1.7,1.85,2.15,2.2,2.25,2.35,2.5,2.6,2.7,2.85,2.95,3.1] roadarea=[
0.09,0.11,0.11,0.14,0.20,0.23,0.23,0.32,0.32,0.34,0.36,0.36,0.38,0.49,0.56,0.59,0.59,0.67,0.69,0.79] #Output passengertraffic=[
5126,6217,7730,9145,10460,11387,12353,15750,18304,19836,21024,19490,20433,22598,25107,33442,36836,40548,42927,43462] freighttraffic=[  1237,1379,1385,1399,1663,1714,1834,4322,8132,8936,11099,11203,10524,11115,13320,16762,18673,20724,20803,21804] # Normalize the original data and add the noise Samplein = Np.mat ([PopulatIon,vehicle,roadarea]) #3 *20 Sampleinminmax = Np.array ([samplein.min]. T.tolist () [0],samplein.max (Axis=1). T.tolist () [0]]). Transpose () #3 * * sampleout = Np.mat ([passengertraffic,freighttraffic]) #2 *20 Sampleoutminmax = Np.array ([Sampleout.min (Axis=1). T.tolist () [0],sampleout.max (Axis=1). T.tolist () [0]]). Transpose () #2 * * Sampleinnorm = (Np.array (Samplein.
T)-sampleinminmax.transpose () [0])/(Sampleinminmax.transpose () [1]-sampleinminmax.transpose () [0])-1). Transpose () Sampleoutnorm = (Np.array (sampleout. T). Astype (float)-sampleoutminmax.transpose () [0])/(Sampleoutminmax.transpose () [1]-sampleoutminmax.transpose () [0  ])-1). Transpose () #initial the parameters maxepochs =1000 learnrate = 0.035 errorfinal = 0.5*10** ( -3) samnum = Indim =  3 Outdim = 2 Hiddenunitnum = 8 W1 = 2*np.random.rand (Hiddenunitnum,indim)-1 B1 = 2*np.random.rand (hiddenunitnum,1)-1 W2 =
    2*np.random.rand (Outdim,hiddenunitnum)-1 b2 = 2*np.random.rand (outdim,1)-1 errhistory = [] for I in range (Maxepochs): Hiddenout = LogSig ((Np.dot (W1,sampleinnorm). Transpose () +b1.transpose ()). Transpose () Networkout = (Np.dot (w2,hiddenout). Transpose () +b2.transpose ()). Transpose () Err = sampleoutnorm-networkout SSE = SUM (SUM (err**2)) #Use the Err O f the whole dataset as the err, rather than one subject, aiming of reduce the Err Fast errhistory.append (SSE) if s
    Se < Errorfinal:break delta2 = Err delta1 = Np.dot (W2.transpose (), Delta2) *hiddenout* (1-hiddenout) DW2 = Np.dot (Delta2,hiddenout.transpose ()) DB2 = Np.dot (Delta2,np.ones ((samnum,1))) Dw1 = Np.dot (Delta1,sampleinno Rm.transpose ()) DB1 = Np.dot (Delta1,np.ones ((samnum,1))) W2 + = Learnrate*dw2 B2 + learnrate*db2 W1 + = Lea 
RNRATE*DW1 B1 + = LEARNRATE*DB1 #For There is a normalization, cacalute the original output use the Min and max value Hiddenout = Logsig ((Np.dot (W1,sampleinnorm). Transpose () +b1.transpose ()). Transpose () Networkout = (Np.dot (W2, Hiddenout). Transpose () +b2.transpose ()). Transpose (diff = sampleoutminmax[:,1]-sampleoutminmax[:,0] networkout2 = (networkout+1)/2 networkout2[0] = networkout2[0]*diff[ 0]+sampleoutminmax[0][0] networkout2[1] = networkout2[1]*diff[1]+sampleoutminmax[1][0] Sampleout = Np.array ( Sampleout) #show The Err curve and the results plt.figure (1) plt.plot (errhistory,label= "error") plt.legend (loc= ' Upper Lef T ') plt.figure (2) plt.subplot (2,1,1) plt.plot (sampleout[0],color= "Blue", linewidth=1.5, linestyle= "-", label= "real
Curve of Passengertraffic ");
Plt.plot (networkout2[0],color= "Red", linewidth=1.5, linestyle= "--", label= "fitting curve"); Plt.legend (loc= ' upper left ') plt.show () Plt.subplot (2,1,2) plt.plot (sampleout[1],color= "Blue", linewidth=1.5,
Linestyle= "-", label= "real curve of freighttraffic");
Plt.plot (networkout2[1],color= "Red", linewidth=1.5, linestyle= "--", label= "fitting curve"); Plt.legend (loc= ' upper left ') plt.show ()

Experimental results:
Error curve

Normalized error curve (iteration 1000 times)
(2) Fit result

Output 1 fit curve (iteration 1000)

Output 1 fit curve (iteration 100,000)
(3) Experimental conclusion
from the error curve in the figure, it can be seen that at the beginning of the training, the error drops very quickly, but as the cumulative error drops to a certain extent, the further decline will be very slow. When the accuracy requirement is high, the number of training times will be increased by using the cumulative error to update the right values. When the sample is small or the accuracy requirement is not high, the accumulative error is used to update the weights, which can greatly accelerate the training speed.