TensorFlow (iv) Realization of elastic network regression algorithm using TensorFlow (multi-linear regression)

The Elastic network regression algorithm is a regression algorithm for synthesizing lasso regression and ridge regression, which can control the effect of single coefficients by adding L1 regular and L2 regular term in loss function.

ImportTensorFlow as TFImportNumPy as NPImportMatplotlib.pyplot as Plt fromSklearnImportdatasetssess=TF. Session ()#loading the iris setiris=Datasets.load_iris ()#petals Length, petal width, calyx width predicted calyx lengthX_vals=np.array ([[X[1],x[2],x[3]] forXinchIris.data]) Y_vals=np.array ([y[0] forYinchIris.data]) Learning_rate=0.001batch_size=50X_data=tf.placeholder (shape=[none,3],dtype=tf.float32) Y_target=tf.placeholder (shape=[none,1],dtype=Tf.float32) A=TF. Variable (Tf.random_normal (shape=[3,1])) b=TF. Variable (Tf.random_normal (shape=[1,1]))#Add linear Model Y=ax+b X*a==>shape (none,1) +b==>shape (none,1)model_out=Tf.add (Tf.matmul (x_data,a), b)#parameterElastic_p1=tf.constant (1.) ELASTIC_P2=tf.constant (1.)#the declaration loss function contains the L1 regular and L2 regular of the slope. #Create a regular iteml1_a_loss=Tf.reduce_mean (Tf.abs (A)) L2_a_loss=Tf.reduce_mean (Tf.square (A)) E1_term=tf.multiply (elastic_p1,l1_a_loss) e2_term=tf.multiply (Elastic_p2,l2_a_loss)#here A is an irregular shape that corresponds to the array form of the 3,1 loss also expands the arrays formLoss=tf.expand_dims (Tf.add (Tf.add (Tf.reduce_mean (Tf.square (y_target-model_out)), e1_term), e2_term), 0)#Initialize Variablesinit=Tf.global_variables_initializer () sess.run (init)#Gradient Descentmy_opt=Tf.train.GradientDescentOptimizer (learning_rate) Train_step=my_opt.minimize (loss)#Loop IterationLoss_rec=[] forIinchRange (1000): Rand_index=np.random.choice (Len (x_vals), size=batch_size)#shape (none,3)rand_x=X_vals[rand_index] Rand_y=Np.transpose ([Y_vals[rand_index]])#RunSess.run (train_step,feed_dict={x_data:rand_x,y_target:rand_y}) Temp_loss=sess.run (loss,feed_dict={x_data:rand_x,y_target:rand_y})#Add a recordloss_rec.append (Temp_loss)#Print    if(i+1)%250==0:Print('Step:%d a=%s b=%s'%(I,str (Sess.run (A)), str (Sess.run (b) )))Print('loss:%s'%str (temp_loss[0]))#Elastic Network Regression Iteration GraphPlt.plot (Loss_rec,'k -', label='Loss') Plt.title ('Loss per Generation') Plt.xlabel ('Generation') Plt.ylabel ('Loss') plt.show ()

TensorFlow (iv) Implement elastic network regression algorithm with TensorFlow (multi-linear regression)