Locally Weighted Linear Regression local weighted linear regression-R implementation

Linear regression is prone to problems of fitting or less fitting.

Local weighted linear regression is a non-parametric learning method, when the new samples are predicted, the new weights are re-trained, and the values of the parameters are obtained by retraining the sample data, each time the parameter value of the prediction is different.

Weight function:

T is used to control the rate of change of weights (recommended for different samples, first by adjusting t values to determine the appropriate T)

Weight function images with different T values:

Local weighted linear regression R implementation:

#Locally Weighted Linear Regression Local weighted regression (non-parametric learning method) # #x为数据矩阵 (mxn m: sample number N: characteristics); Y observation value (MX1); XP is the sample feature that needs to be predicted, the weight change rate of the T weight function # The error termination condition, the amplitude of the next two search results, #step为设定的固定步长; maxiter maximum number of iterations, Alpha,beta is the parameter lwlregression<-function of the backtracking descent method (X,y,xp,t,error , maxiter,stepmethod=t,step=0.001,alpha=0.25,beta=0.8) {w<-exp ( -0.5* (X-XP) ^2./t^2) #权重函数, W (i) represents the weight of the first sample point, T control weight Change rate m<-nrow (x) x<-cbind (Matrix (1,m,1), x) N<-ncol (x) Theta<-matrix (0,n,1) #theta初始值都设置为0 iter<- 0 newerror<-1 while ((Newerror>error) | (    Iter<maxiter) {iter<-iter+1 H<-x%*%theta des<-t (t (w* (H-Y))%*%x) #梯度 #回溯下降法求步长t if (stepmethod==t) { Step=1 new_theta<-theta-step*des New_h<-x%*%new_theta costfunction<-t (w* (h-y))%*% (h-y) # (least squares loss function) Bureau Part weighted linear regression loss function new_costfunction<-t (w* (new_h-y))%*% (new_h-y) #回溯下降法求步长step while (new_costfunction>costfunction- Alpha*step*sum (Des*des)) {Step<-step*beta new_theta<-theta-step*des New_h<-x%*%new_theta new_ CostfunctiOn<-t (w* (new_h-y))%*% (new_h-y)} newerror<-t (Theta-new_theta)%*% (Theta-new_theta) theta<-new_ Theta} #直接设置固定步长 if (stepmethod==f) {new_theta<-theta-step*des newerror<-t (Theta-new_theta )%*% (Theta-new_theta) Theta<-new_theta}} xp<-cbind (1,XP) Yp<-xp%*%theta #costfunction <-t (w* (x%*%t Heta)-y)%*% (x%*%theta-y) #result <-list (yp,theta,iter,costfunction) #names (Result) <-c (' Fit value ', ' coefficients ', ' iteration count ', ' ERROR ') #result YP}

　　

Using local linear weighted regression to predict the Y value of each sample point x, a smoothing curve is obtained after connecting each predicted value, reflecting the nonlinear relationship between Y and X.

> t (x) [, 1] [, 2] [, 3] [, 4] [, 5] [, 6] [, 7] [, 8] [, 9] [, 10] [, 11] [, 12] [, 13] [, 14] [, 15][1,] 58 59 60 61 6 2 x (4) [, 5] [, 6] [, 7] [, 8] [, + 9] [, 10] [, 3] [72>] [, 2] [, 11] [, 12] [, 13] [, 14] [, 15]  [1,] 111-121 123 131, 136, 142 145 147 151 148 151 148>> LM (Y~X) call:lm (formula = y ~ x) coefficients: (Intercept) x-50.245 2.864 > Yy<--50.245+2.864*x> t (yy) [, 1] [ , 2] [, 3] [, 4] [, 5] [, 6] [, 7] [, 8] [, 9] [, 10] [, 11] [, 12] [, 13] [, 14] [, 15][1,] 115.867 11 8.731 121.595 124.459 127.323 130.187 133.051 135.915 138.779 141.643 144.507 147.371 150.235 153.099 155.963>> g&lt ;-apply (X,1,function (XP) {lwlregression (x,y,xp,3,1e-7,100000,stepmethod=f,step=0.00001,alpha=0.25,beta=0.8)})    >> T (g) [, 1] [, 2] [, 3] [, 4] [, 5] [, 6] [, 7] [, 8] [, 9] [, 10] [, 11] [, 12] [, 13]  [, 14] [1,] 116.093 119.0384 122.1318 125.3421 128.6115 131.862 135.009 137.9771 140.7136 143.194 145.4244 147.4373 149.2831 151. 018 [, 15][1,] 152.693>> plot (X,y,pch=20,xlim=c (57,73), Ylim=c (109,157)) > lines (x,y,col= ' green ') > lines (x , yy,col= ' Blue ') > points (x,g,pch=21) > lines (x,g,col= ' red ') > Legend ("BottomRight", Legend=c (' Scatter chart ', ' Fit straight line ', '  Weighted scatter plot '), Lwd=1,col=c (' green ', ' blue ', ' red ') >

　　

Locally Weighted Linear Regression local weighted linear regression-R implementation