Algorithm #03--specifically explains the principle and code of least squares

Principle of least squares method

The goal of the least squares method is to find the minimum sum of squares of errors, and there are two corresponding types: linear and nonlinear. The solution of the linear least squares is closed-form (for example, the following), and the nonlinear least squares is not closed-form, and the pass is often solved using iterative method (such as the Gauss-Newton iterative method, this article is not introduced).

"First get a linear equation group"

1. Concept

Least squares (also known as the least squares method) is a mathematical optimization technique. It matches by minimizing the squared error and finding the best function of the data.

Using the least squares method, the unknown data can be obtained easily, and the sum of the errors between the obtained data and the actual data is minimized.

The least squares can also be used for curve fitting.

2. Principle

Function Prototypes:

Known:

(X0,Y0). (x1. Y1) ... (Xi,yi) ... (Xn,yn) a point, n>=k.

Deviation squared sum:

The squared and minimum deviations can be obtained by making the partial derivative equal to zero:

The simplified left equation has:

Written in matrix form: formula ①

This Vandermonde matrix simplification can be obtained: the formula ②

That is to say x*a=y, then a = (X ' *x) -1*x ' *y, we get the coefficient matrix A, at the same time, we also get the fitting curve.

Gaussian elimination element method

"Then solve the linear equations, that is, the formula ①"

1. Concept

Mathematically, the Gaussian elimination method (or the Gaussian elimination method) (English: Gaussian elimination) is an algorithm in linear algebra that can be used to solve a linear equation, to find the rank of a matrix, and to find the inverse matrix of a reversible square. When used with a matrix. Gaussian elimination method will produce a "line ladder".

2. Principle

3. Pseudo-code

This algorithm is a little different from the one mentioned above. It starts with the largest part of the absolute value. This can improve the stability of the algorithm.

The algorithm is computed from left to right. Skip to the next column and row for each of the following three steps:

• A number of the maximum absolute value of the I column is determined, and the value of line I is exchanged with the row, so that the row has the maximum of the column;
• Divide the number of I columns by that number so that the number of rows I is 1.
• All elements under Line (I+1) (including line (j+1)) are converted to 0.

After all the steps have been completed, the matrix becomes a row ladder matrix, which can be solved by substituting the method.

i =1j =1  while(i≤m andJ≤N) DoFind PivotinchColumn J, startinginchRow I//start from line I. Find the maximum value in column J (i, J values should remain unchanged)Maxi = i forK = i+1  toM Do     if ABS(A[k,j]) >ABS(A[maxi,j]) ThenMaxi = k//Use the Swap method to find the maximum (absolute value maximum)     End If    End   for   ifA[maxi,j]≠0  Then //Determines whether the absolute maximum value found is zero: if it is not zero, the following operation is performed; if zero, the following (i+1) line (including line i+1) is zero, and no further processing is required to jump directly to the line (i+1) of subsection (j+1)Swap rows I andMaxi, but Do  notChange the value  ofI//Exchange The line I with the largest value found, keeping the I value unchanged (I value records the starting line of this operation)Now A[i,j] 'll contain theOldvalue  ofA[MAXI,J].Divide  eachEntryinchRow I byA[I,J]//The first line of the interchange is normalized (all elements of line I are divided by a[i,j])Now A[i,j] would have the value 1.      forU = i+1  toM Do   in Column J, (I+1) below (with line (i+1)) all elements minus a[i,j] until the i+1 line in column J is 0       SubtractA[U,J] * Row I fromRow u now a[u,j] 'll be0, since A[u,j]-a[i,j] * a[u,j] = a[u,j]-1* A[u,j] =0.      End   fori = i +1      End Ifj = j +1 //In column J. All elements below (i+1) line (including line (i+1)) are zero.
Move to Column (j+1), starting with line (I+1) to repeat the above steps.
 end while

Code

public class CurveFitting { ///<summary> ///最小二乘法拟合二元多次曲线 ///比如y=ax+b ///当中MultiLine将返回a。b两个參数。
 //  a corresponding multiline[1] ///  b corresponding Multiline[0] //  </summary>  ///  <param name= "Arrx" > x-coordinate set of known points </param>  ///  <param name= "Arry" > known point y-coordinate set </param>  ///  <param name= "Length" > number of known points </param>  ///  <param name= "Dimension" > maximum number of equations </param>   Public Static Double[]MultiLine(Double[] Arrx,Double[] Arry,intLengthintDimension) {intn = dimension +1;dimension+1 coefficients must be required for//dimension quadratic equation Double[] Guass =New DoubleN [n +1]; for(inti =0; I < n; i++) {//Find matrix formula ① intJ for(j =0; J < N; J + +) {Guass[i][j] = Sumarr (Arrx, j + i, length);//Formula ① equal to the first matrix on the left, i.e. a in ax=b} Guass[i][j] = Sumarr (Arrx, I, Arry,1, length);//Formula ① The matrix to the right of the equal sign, or B in Ax=b}returnComputgauss (Guass, N);//Gauss elimination element method}//to the n-th-square of the elements of an array, i.e. the elements in matrix a Private Static Double Sumarr(Double[] arr,intNintLength) {Doubles =0; for(inti =0; i < length; i++) {if(Arr[i]! =0|| n! =0) {s = s + Math.pow (Arr[i], n); }Else{s = s +1; } }returnS }//to the n-th-square of the elements of an array, i.e. the elements in matrix B Private Static Double Sumarr(Double[] arr1,intN1,Double[] arr2,intN2,intLength) {Doubles =0; for(inti =0; i < length; i++) {if((arr1[i]! =0|| N1! =0) && (arr2[i]! =0|| N2! =0) s = s + Math.pow (Arr1[i], N1) * MATH.POW (Arr2[i], N2);Elses = s +1; }returnS }//Gaussian elimination method for solving linear equations Private Static Double[]Computgauss(Double[] Guass,intN) {intI, J;intK, M;DoubleTempDoubleMaxDoubleSDouble[] x =New Double[n]; for(i =0; I < n; i++) {X[i] =0.0;//Initialize} for(j =0; J < N; J + +) {max =0; K = J;//Starting from line I, find the maximum value in column J (i, J value should remain unchanged)  for(i = j; i < n; i++) {if(Math.Abs (guass[i][j]) > Max) {max = guass[i][j];//Use the Swap method to find the maximum (absolute value maximum)K = i; } }if(k! = j) {//Swap line J with the row where the maximum value is found. Keep I value unchanged (j Value records the starting line of this operation)  for(M = j; m < n +1; m++) {temp = guass[j][m]; GUASS[J][M] = guass[k][m]; GUASS[K][M] = temp; } }if(max = =0) {//"This linear equation is a mysterious linear equation" returnX }in column M, (j+1) below (with line (j+1)) all elements minus guass[j][m] * S/(Guass[j][j]) //until the I+1 line in column M is zero  for(i = j +1; I < n; i++) {s = guass[i][j]; for(M = j; m < n +1; m++) {Guass[i][m] = Guass[i][m]-guass[j][m] * S/(guass[j][j]); } } }//End for (j=0;j<n;j++) //back-generation process (see formula 4.1.5)  for(i = n-1; I >=0; i--) {s =0; for(j = i +1; J < N; J + +) {s = s + guass[i][j] * X[j]; } X[i] = (Guass[i][n]-s)/guass[i][i]; }returnX }the//return value is the coefficient of the function  Public Static void Main(string[] args) {Double[] x = {0,1,2,3,4,5,6,7};Double[] y = {0,1,4,9, -, -, $, the};Double[] A = MultiLine (x, Y,8,2); for(inti =0; I <a.length;i++) {System. out. println (A[i]); } } }

Output:

0.708333333333342
-0.37500000000000583
1.0416666666666674

Take the whole to get y=x^2.

Algorithm #03--specifically explains the principle and code of least squares