Iris Classification Neural Network

Iris Classification Neural Networkneural Network

Formula derivation

\[\begin{align}a & = x \cdot w_1 \y & = a \cdot w_2 \& = x \cdot w_1 \cdot w_2 \y & = Softmax (y) \end{align }\]

Code (training only)

\[a = x \cdot w_1 \y = a \cdot w_2\]

= tf.Variable(tf.random_normal([4,5], stddev=1, seed=1= tf.Variable(tf.random_normal([5,3], stddev=1, seed=1= tf.placeholder(tf.float32, shape=(None4), name=‘x-input‘== tf.matmul(a, w2)

Since there is supervised learning, it is necessary to give a label at the training stage to calculate the cross-entropy

# 用来存储数据的标签= tf.placeholder(tf.float32, shape=(None3), name=‘y-input‘)

The activation function of the hidden layer is sigmoid

= tf.sigmoid(y)

Softmax and Cross entropy (Corss entropy), the loss function is the mean value of cross entropy

# softmax & corss_entropy= tf.nn.softmax_cross_entropy_with_logits_v2(labels=y_, logits=y)# mean= tf.reduce_mean(cross_entropy)

In order to prevent the neural network from overfitting, it is necessary to join the regularization term and generally choose "L2 regularization".

=+\    +\    tf.contrib.layers.l2_regularizer(regulation_lamda)(w2)

In order to accelerate the training process of neural network, we need to add "exponential attenuation" technique.

It represents the calculation diagram of the training process, and the optimization method chooses Adam algorithm, which is essentially the inverse propagation algorithm. You can also choose the gradient descent method (Gradientdescentoptimizer)

= tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)

Training phase

 withTf. Session () asSess:# Session is best opened in "context mechanism" to prevent resource leaksInit_op=Tf.global_variables_initializer ()# Initialize the parameters of the nodes in the network, mainly W1,W2Sess.run (INIT_OP) steps= 10000     forIinch Range(steps): Beg=(I*Batch_size)%Dataset_size# Calculate BatchEnd= min(Beg+Batch_size, Dataset_size)# Calculate BatchSess.run (Train_step, feed_dict={X:x[beg:end], Y_:y[beg:end]})# Reverse propagation, training network        ifI%  + == 0: total_corss_entropy=Sess.run (# Calculate Cross entropyCross_entropy_mean,# Calculate Cross entropyFeed_dict={x:x, y_:y}# Calculate Cross entropy)Print(" after%dtraining steps, cross entropy on all data is%g" %(i, total_corss_entropy))

In the training phase, a "sliding average model" is needed to improve the robustness of the model on the test data (this is the book's statement, which I think is generalization capability).

All code

#-*-Encoding=utf8-*- fromSklearn.datasetsImportLoad_irisImportTensorFlow asTfdefLabel_convert (Y): l= List() forYinchY:ifY== 0: L.append ([1,0,0])elifY== 1: L.append ([0,1,0])elifY== 2: L.append ([0,0,1])returnLdefLoad_data (): Iris=Load_iris () X=Iris.data Y=Label_convert (Iris.target)return(x, Y)if __name__ == ' __main__ ': X, Y=Load_data () learning_rate= 0.001Batch_size= TenDataset_size=  MaxRegulation_lamda= 0.001W1=Tf. Variable (Tf.random_normal ([4,5], StdDev=1, Seed=1)) W2=Tf. Variable (Tf.random_normal ([5,3], StdDev=1, Seed=1)) x=Tf.placeholder (Tf.float32, shape=(None,4), name=' X-input ') Y_=Tf.placeholder (Tf.float32, shape=(None,3), name=' Y-input ') A=Tf.matmul (x, W1) y=Tf.matmul (A, W2) y=Tf.sigmoid (y) cross_entropy=TF.NN.SOFTMAX_CROSS_ENTROPY_WITH_LOGITS_V2 (Labels=Y_, Logits=Y) Cross_entropy_mean=Tf.reduce_mean (cross_entropy) loss=Cross_entropy_mean+ \Tf.contrib.layers.l2_regularizer (REGULATION_LAMDA) (W1)+ \Tf.contrib.layers.l2_regularizer (REGULATION_LAMDA) (W2) Train_step=Tf.train.AdamOptimizer (learning_rate=learning_rate). Minimize (loss) withTf. Session () asSess:init_op=Tf.global_variables_initializer () Sess.run (init_op) steps= 10000         forIinch Range(steps): Beg=(I*Batch_size)%Dataset_size End= min(Beg+Batch_size, Dataset_size) Sess.run (Train_step, feed_dict={X:x[beg:end], Y_:y[beg:end]})ifI%  + == 0: total_corss_entropy=Sess.run (Cross_entropy_mean, feed_dict={x:x, y_:y})Print(" after%dtraining steps, cross entropy on all data is%g" %(i, total_corss_entropy))Print(Sess.run (W1))Print(Sess.run (W2))

Experiment Result

Random split Cross validation

Iris Classification Neural Network