TensorFlow Learning Notes (5)--Realization of convolution neural network (mnist dataset)

this uses TensorFlow to implement a simple convolution neural network using mnist datasets. The network structure is: Data input layer – convolution layer-----------------------------------------------------------

Import TensorFlow as TF import numpy as NP import input_data mnist = input_data.read_data_sets (' data/', one_hot=true) pri NT ("Mnist ready") Sess = tf. InteractiveSession () # defines the initialization function for reuse. Make some random noises to the weights to break the full symmetry, using the truncated normal distribution, the standard deviation is set to 0.1, # and because of the use of Relu, it also adds some small positive values (0.1) To avoid death nodes (dead neurons) def weight_variable ( SHAPE): initial = Tf.truncated_normal (shape, stddev=0.1) return TF. Variable (initial) def bias_variable (shape): initial = Tf.constant (0.1, Shape=shape) return TF. Variable (initial) def conv2d (X, W): Return tf.nn.conv2d (x, W, strides=[1, 1, 1, 1], padding= ' SAME ') # parameters specify the size of the convolution core, multiple The number of fewer channel and filter numbers is the number of feature graphs # 2x2 maximum pooling, which is to reduce a 2x2 pixel block to 1x1 pixels.
Maximum pooling preserves the pixel with the highest grayscale value in the original pixel block, which preserves the most significant features. def max_pool_2x2 (x): Return Tf.nn.max_pool (x, Ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding= ' SAME ') N_input = 78 4 # 28*28 Grayscale, pixel number 784 n_output = 10 # is 10 classification problem # before designing the network structure, define the input placeholder,x is the feature, Y is the real label x = Tf.placeholder (tf.float32 , [None, N_input]) y = Tf.placeholder (Tf.float32, [None,N_output] X_image = Tf.reshape (x, [-1, 28, 28, 1]) # to preprocess the image, convert the input vector of 1 D to 2D image structure, i.e. 1*784 to 28*28 structure,-1 represents the sample number is not fixed, 1 represents the number of color channels # Define the first convolution layer, using the previously written function for parameter initialization, including weight and bias w_conv1 = Weight_variable ([3, 3, 1,]) B_CONV1 = bias_variable ([+]) H_conv1 = t  F.nn.relu (conv2d (X_image, W_CONV1) + b_conv1) H_pool1 = max_pool_2x2 (H_CONV1) # defines the second convolution layer w_conv2 = weight_variable ([3, 3, ] B_conv2 = Bias_variable ([)] H_conv2 = Tf.nn.relu (conv2d (h_pool1, w_conv2) + b_conv2) H_pool2 = max_pool_2x2 (h_ CONV2) # FC1, converts the two-second 7*7 to a total of 128 feature maps to 1D vectors, implying that node 1024 is defined by itself W_FC1 = Weight_variable ([7*7*64, 1024]) B_FC1 = Bias_variable ([ 1024]) H_pool2_flat = Tf.reshape (H_pool2, [-1, 7*7*64]) H_fc1 = Tf.nn.relu (Tf.matmul (H_pool2_flat, W_FC1) + b_fc1) # to Lessen Cross-fitting, using dropout layer Keep_prob = Tf.placeholder (tf.float32) H_fc1_drop = Tf.nn.dropout (H_FC1, Keep_prob) # Dropout layer output connects a Softmax layer, obtains the final probability output W_FC2 = weight_variable ([1024, ten]) B_FC2 = Bias_variable ([ten]) pred = Tf.nn.softmax (tf . Matmul (H_fc1_drop, W_FC2) + b_fc2) #前向传播的预测值, PrinT ("CNN READY") # define loss function for cross entropy loss function cost = Tf.reduce_mean (-tf.reduce_sum (Y*tf.log (pred), reduction_indices=[1)) # Optimizer OPTM = Tf.train.AdamOptimizer (0.001). Minimize (cost) # defines the operation of the evaluation accuracy Corr = Tf.equal (Tf.argmax (pred, 1), Tf.argmax (Y, 1)) # Comparison of the predicted value of the cable The index of the real label is the same, return true, not return false accuracy = Tf.reduce_mean (Tf.cast (Corr, Tf.float32)) # Initialize all parameters Tf.global_ Variables_initializer (). Run () print ("Functions READY") Training_epochs = 1000 # All samples are iterated 1000 times batch_size = 100 # each Iteration selection 1
    00 Samples Display_step = 1 for i in Range (training_epochs): avg_cost = 0. total_batch = Int (mnist.train.num_examples/batch_size) batch = Mnist.train.next_batch (batch_size) Optm.run (feed_di Ct={x:batch[0], y:batch[1], keep_prob:0.7}) Avg_cost + = Sess.run (cost, feed_dict={x:batch[0), y:batch[1], keep_prob:1. 0})/total_batch if I% display_step ==0: # 10 times per training, test the accuracy rate train_accuracy = Accuracy.eval (feed_dict={x:batch [0], y:batch[1], keep_prob:1.0}) test_accuracy = Accuracy.eval (feed_dict={x:mnist.test.images, Y:mnist.test.labels, keep_prob:1.0}) print ("Step:%d cost:%.9f TRAIN accuracy:%.3f TES
 T accuracy:%.3f "% (I, Avg_cost, Train_accuracy, test_accuracy))" Print ("Done")

After training iterations 1000 times, the test classification accuracy rate reached 98.6%

step:999  cost:0.000048231  TRAIN accuracy:0.990  TEST accuracy:0.986

Reached 99.1% at 2000 times.

step:2004  cost:0.000042901  TRAIN accuracy:0.990  TEST accuracy:0.991

Compared with the simple neural network, CNN has a good effect, and the main performance improvement comes from the better network design, namely convolution neural network to extract and abstract the image features. Based on the weight sharing of the convolution kernel, the parameters of CNN are not exploded, and the calculation quantity is reduced, so the performance of the whole model is greatly improved.