An example of using TensorFlow to implement the Deming regression algorithm

This article mainly introduces the use of TensorFlow implementation of the Deming regression algorithm example, has a certain reference value, and now share to everyone, the need for friends can refer to

If the least squares linear regression algorithm is minimized to the vertical distance of the regression line (that is, parallel to the y-axis direction), the Deming regression is minimized to the total distance of the regression line (that is, perpendicular to the regression line). It minimizes the X-value and the Y-value two-direction error, the specific contrast Turu.

The difference between the linear regression algorithm and the Deming regression algorithm. The linear regression on the left minimizes the vertical distance to the regression line, and the right Deming regression minimizes the total distance to the regression line.

The loss function of the linear regression algorithm minimizes the vertical distance, and it needs to minimize the total distance. Given the slope and intercept of a line, there is a known geometric formula for solving the vertical distance from a point to a line. Substituting a geometric formula and minimizing the distance of the TensorFlow.

The loss function is a geometric formula consisting of numerator and denominator. Given a line y=mx+b, point (X0,Y0), the formula for the distance between the two is:

# deming returns #----------------------------------# # This function shows how to use TensorFlow to# solve linear Deming regression.# y = Ax + b## We'll use the iris data, specifically:# y = sepal length# x = Petal Widthimport Matplotlib.pyplot as Pltim Port NumPy as Npimport TensorFlow as Tffrom sklearn import datasetsfrom tensorflow.python.framework import Opsops.reset_de Fault_graph () # Create graphsess = tf. Session () # Load the data# iris.data = [(sepal length, sepal width, petal length, petal Width)]iris = Datasets.load_iris () x _vals = Np.array ([x[3] for x in iris.data]) y_vals = Np.array ([y[0] for y in Iris.data]) # Declare Batch sizebatch_size = 50 # Initialize Placeholdersx_data = Tf.placeholder (Shape=[none, 1], dtype=tf.float32) Y_target = Tf.placeholder (shape=[ None, 1], dtype=tf.float32) # Create variables for linear Regressiona = TF. Variable (Tf.random_normal (shape=[1,1])) b = tf. Variable (Tf.random_normal (shape=[1,1)) # Declare Model operationsmodel_output = Tf.add (Tf.matmul (X_data, A), b) # DECLis demming loss functiondemming_numerator = Tf.abs (Tf.subtract (Y_target, Tf.add (Tf.matmul (X_data, A), b))) Demming_ Denominator = Tf.sqrt (Tf.add (Tf.square (A), 1)) loss = Tf.reduce_mean (Tf.truep (Demming_numerator, Demming_denominator) ) # Declare optimizermy_opt = Tf.train.GradientDescentOptimizer (0.1) Train_step = My_opt.minimize (loss) # Initialize Variablesinit = Tf.global_variables_initializer () sess.run (init) # Training Looploss_vec = []for i in range]: Rand_ind ex = Np.random.choice (Len (x_vals), size=batch_size) rand_x = Np.transpose ([X_vals[rand_index]]) rand_y = Np.transpose ([ Y_vals[rand_index]] Sess.run (Train_step, feed_dict={x_data:rand_x, y_target:rand_y}) Temp_loss = Sess.run (loss, feed _dict={x_data:rand_x, y_target:rand_y}) loss_vec.append (Temp_loss) if (i+1)%50==0:print (' Step # ' + str (i+1) + ' A = ' + str (sess.run (A)) + ' B = ' + str (sess.run (b))) print (' Loss = ' + str (temp_loss)) # Get the optimal COEFFICIENTS[SL Ope] = Sess.run (A) [y_intercept] = Sess.run (b) # GeT best fit Linebest_fit = []for i in X_vals:best_fit.append (slope*i+y_intercept) # Plot The Resultplt.plot (X_vals, Y_vals, ' O ', label= ' Data Points ') plt.plot (x_vals, Best_fit, ' R ', label= ' best fit line ', linewidth=3) plt.legend (loc= ' Upper Left ') plt.title (' Sepal length vs pedal width ') plt.xlabel (' pedal width ') plt.ylabel (' Sepal Length ') plt.show () # Plot loss Over Timeplt.plot (Loss_vec, ' K ') plt.title (' L2 loss per Generation ') Plt.xlabel (' Generation ') Plt.ylabel (' L2 loss ') Plt.show ()

Results:

In this paper, the Deming regression algorithm and the linear regression algorithm are basically consistent with the results. The key difference between the two is the loss function metric between the predicted value and the data point: the loss function of the linear regression algorithm is the vertical distance loss, and the Deming regression algorithm is the vertical distance loss (the total distance loss to the X and Y axes).

Note that the implementation type of the Deming regression algorithm here is the overall regression (total least squares error). The overall regression algorithm assumes that the error of the X value and the Y value is similar. We can also use different errors to expand the x-axis and y-axis distance calculations based on different concepts.