Least squares and Python implementations

Summary of reference least squares machine learning: How to use least squares in Python

What is "least squares"?

Definition: Least squares (also known as the least squares method) is a mathematical optimization technique that matches by minimizing the squared error and finding the best function of the data.

Function: Using the least squares method, the unknown data can be obtained easily, and the sum of the errors between the obtained data and the actual data is minimized.

Principle: The linear position is determined by " residual squared and minimum " (in mathematical statistics, the residuals are the difference between the actual observed value and the estimated value).

Mathematical formula:

Basic idea: For a unary linear regression model, assume that n groups of observations (X1,y1), (X2,y2), ..., (Xn,yn) are obtained from the population, and that for these n points in the plane, you can use an infinite number of curves to fit. Linear regression requires that the sample regression function fit the set of values as well as possible, that is, the line should be as close to the center of the sample data as possible. Therefore, the criteria for selecting the best fit curve can be determined as follows: The total fit error (i.e. total residuals) is minimized . For the minimum, the solution is solved by the simultaneous equations of the parameters respectively.

• The least square method is the direct use of the minimization of the error level and the derivation of the parameter, the solution of the parameter, which belongs to the relatively determined value
• The gradient descent method, which belongs to the iterative method, knows that the gradient is descending in the direction of Liu, one by one to iterate.

1. The principle of least squares and the problems to be solved

2. Algebraic method for least squares

3. Limitations and applicable scenarios of least squares

##最小二乘法ImportNumPy as NP##科学计算库ImportSciPy as SP##在numpy基础上实现的部分算法库ImportMatplotlib.pyplot as Plt##绘图库 fromScipy.optimizeImportLeastsq##引入最小二乘法算法" "set up sample data, real data needs to be handled here" "##样本数据 (xi,yi), you need to convert the form of an array (list)Xi=np.array ([6.19,2.51,7.29,7.01,5.7,2.66,3.98,2.5,9.1,4.2]) Yi=np.array ([5.25,2.83,6.41,6.71,5.1,4.23,5.05,1.98,10.5,6.3])" "set the shape determination process of the fitting function and the deviation function function: 1. Draw the sample image first 2. Determine the function form (line, parabolic, sine cosine, etc.) according to the approximate shape of the sample image" "##需要拟合的函数func: Specifying the shape of a functiondeffunc (p,x): K,b=Preturnk*x+b##偏差函数: x, Y are the list: here's X, Y, and one by one in the Xi,yi .deferror (p,x,y):returnFunc (p,x)-y" "Main parts: The accompanying section describes the return value of the 1.LEASTSQ function tuple, the first element is the solution result, the second is the cost value of the solution (personal Understanding) 2. The official website's exact words (second value): Value of the price function at the Solution 3. Instance:para=> (Array ([0.61349535, 1.79409255]), 3) 4. The number of the first value in the return value tuple is consistent with the number of parameters that need to be solved" "#k,b The initial value, can be arbitrarily set, after several experiments, found that the value of p0 will affect the cost value: Para[1]p0=[1,20]#package parameters other than P0 in the error function into args (use required)PARA=LEASTSQ (error,p0,args=(xi,yi))#Read Resultsk,b=Para[0]Print("k="K"b=", B)Print("Cost :"+str (para[1]))Print("the fitted line for the solution is:")Print("y="+str (Round (k,2)) +"x+"+str (Round (b,2)))" "draw to see the fit effect. Matplotlib default does not support Chinese, label set Chinese words need to be set separately if error, change into English can" "#Draw Sample PointsPlt.figure (figsize= (8,6))##指定图像比例: 8:6Plt.scatter (xi,yi,color="Green", label="Sample Data", linewidth=2) #draw a line in a fitX=np.linspace (0,12,100)##在0-15 draw 100 consecutive points directlyY=k*x+b##函数式Plt.plot (x,y,color="Red", label="Fit Straight Line", linewidth=2) plt.legend (Loc='Lower Right')#Draw LegendPlt.show ()

The results are as follows:

Output Result:

k= 0.900458420439 b= 0.831055638877
Cost:1
The fitted line for the solution is:
y=0.9x+0.83

Drawing results:

Least squares and Python implementations