TensorFlow Training Mnist DataSet (3)--convolutional neural network

The accuracy of the mnist test set is about 90% and 96%, respectively, for single-layer neural networks and multilayer neural networks in the previous two essays. The correct rate has been greatly improved after the multi-layer neural network has been swapped. This time the convolutional neural network will be used to continue the test.

1. Basic structure of the model

As shown, there are 8 layers (including the dropout layer) in the model used this time. The convolution layer and the pool layer each have two layers.

In the whole model, the input layer is responsible for the data input, the convolution layer is responsible for extracting the feature of the picture, the pool layer adopts the maximum pooling method, highlights the main features, and reduces the parameter dimension; The dropout layer can reduce the computational amount of each training and can avoid overfitting problem to some extent. The final output layer then synthesizes the data of each feature to obtain the final result.

The start of the dropout layer does not increase the training parameters, but only randomly disconnects the connection arcs between certain nodes, so that they do not participate in the training for the time being.

2. Data preprocessing

The data used for training is read first.

 from Import  = input_data.read_data_sets ('./data/mnist', one_hot=true)

In the preceding input layer, each sample entered is one-dimensional data, and the sample data of the convolutional neural network will be multidimensional. So we need to reshape the data we read to meet the requirements.

Data.reshape ([BatchSize, 28, 28, 1])

3. Input Layer

The shape of the input layer is: bitchsize * 28 * 28 * 1, the first parameter represents the number of samples per Mini-batch, by passing in none can let TensorFlow automatically infer, the following three parameters indicate that each sample has a height of 28, a width of 28, and a channel count of 1.

Inputlayer = Tf.placeholder (Tf.float32, Shape=[none, 28, 28, 1])

4. Convolution layer

The convolution core size of the first convolutional layer is 5 * 5 and the number is 32. The padding method uses ' same '. The second convolution layer is similar, only the number of channels, the output dimension is not the same.

1 convFilter1 = tf. Variable (Tf.truncated_normal ([5, 5, 1, +], mean=0, stddev=0.1))2 convBias1   = tf. Variable (Tf.truncated_normal ([3], mean=0, stddev=0.1)) convLayer1  = tf.nn.conv2d (input= Inputlayer, Filter=convfilter1, strides=[1, 1, 1, 1], padding='same') 4 convLayer1  = tf.add (convLayer1, CONVBIAS1)5 convLayer1  = Tf.nn.relu ( ConvLayer1)

5. Pool Layer

The size of the sliding window is 2 * 2, and the sliding stride is 2 on the high and wide dimensions, and the other dimension is 1. The second pool layer in this model is the same as the first pooled layer.

PoolLayer1 = Tf.nn.max_pool (Value=convlayer1, Ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='same
   
     ')
   

6. Fully connected Layer

The full join layer expands the multidimensional features previously extracted in each sample into one dimension, as input to the full join layer.

1 fullweight = tf. Variable (Tf.truncated_normal (SHAPE=[7 * 7 *, 1024x768], mean=0, stddev=0.1))2 fullbias   = tf. Variable (Tf.truncated_normal (shape=[1024], mean=0.0, stddev=0.1))3 fullinput  = Tf.reshape ( PoolLayer2, [-1, 7 * 7 * +])4 fulllayer  = tf.add (Tf.matmul (Fullinput, fullweight), Fullbias)  5 fulllayer  = Tf.nn.relu (Fulllayer)

7, dropout layer

The dropout layer can prevent overfitting problems. The reserved rate specified here is 0.8.

Droplayer = Tf.nn.dropout (Fulllayer, keep_prob=0.8)

8. Output Layer

The final output is a 10-digit classification.

1 outputweight = tf. Variable (Tf.truncated_normal (shape=[1024, ten], mean=0.0, stddev=0.1))2 outputbias   = tf. Variable (Tf.truncated_normal (shape=[10], mean=0, stddev=0.1))3 outputlayer  = Tf.add ( Tf.matmul (Droplayer, outputweight), Outputbias)

The rest of the model is similar to the previous multilayer neural network, which is no longer described here.

9. Model's performance on training set and test set

As can be seen from the model diagram, the complexity of this model is much higher than that of the previous multilayer neural network. Because of this, each iteration is more time-consuming than the previous one, which takes more than 1500 times a single time. Although only a few layers have been added (of course, in addition to the increase in the number of layers), but the increase in computational capacity is very much.

The following two graphs show the output of the previous and subsequent iterations of the convolutional neural network, and it is found that the final convolutional neural network is close to 100% accuracy in the training set.

The accuracy on the test set has also reached 98% to 99%, which provides about 2% more than a multilayer neural network.

Report:

The complete code is as follows:

1 ImportTensorFlow as TF2  fromTensorflow.examples.tutorials.mnistImportInput_data3 Import Time4 5 #reading Data6Mnist = Input_data.read_data_sets ('./data/mnist', one_hot=True)7 8 #Input Layer9Inputlayer = Tf.placeholder (Tf.float32, Shape=[none, 28, 28, 1])Ten  One #convolution layer (1) AConvFilter1 = tf. Variable (Tf.truncated_normal ([5, 5, 1, +], mean=0, stddev=0.1)) -CONVBIAS1 = tf. Variable (Tf.truncated_normal ([+], mean=0, stddev=0.1)) -ConvLayer1 = tf.nn.conv2d (Input=inputlayer, Filter=convfilter1, strides=[1, 1, 1, 1], padding='same') theConvLayer1 =Tf.add (ConvLayer1, CONVBIAS1) -ConvLayer1 =Tf.nn.relu (convLayer1) -  - #pool Layer (1) +PoolLayer1 = Tf.nn.max_pool (Value=convlayer1, Ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='same') -  + #convolution layer (2) AConvFilter2 = tf. Variable (Tf.truncated_normal ([5, 5, +, +], mean=0, stddev=0.1)) atCONVBIAS2 = tf. Variable (Tf.truncated_normal ([up], mean=0, stddev=0.1)) -ConvLayer2 = tf.nn.conv2d (Input=poollayer1, Filter=convfilter2, strides=[1, 1, 1, 1], padding='same') -ConvLayer2 =Tf.add (ConvLayer2, CONVBIAS2) -ConvLayer2 =Tf.nn.relu (convLayer2) -  - #pool Layer (2) inPoolLayer2 = Tf.nn.max_pool (Value=convlayer2, Ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='same') -  to #fully connected layer +Fullweight = tf. Variable (Tf.truncated_normal (SHAPE=[7 * 7 *, 1024x768), mean=0, stddev=0.1)) -Fullbias = tf. Variable (Tf.truncated_normal (shape=[1024), mean=0.0, stddev=0.1)) theFullinput = Tf.reshape (PoolLayer2, [-1, 7 * 7 * 64]) *Fulllayer =Tf.add (Tf.matmul (Fullinput, fullweight), Fullbias) $Fulllayer =Tf.nn.relu (Fulllayer)Panax Notoginseng  - #Dropout Layer theDroplayer = Tf.nn.dropout (Fulllayer, keep_prob=0.8) +  A #Output Layer theOutputweight = tf. Variable (Tf.truncated_normal (shape=[1024), mean=0.0, stddev=0.1)) +Outputbias = tf. Variable (Tf.truncated_normal (shape=[10), mean=0, stddev=0.1)) -Outputlayer =Tf.add (Tf.matmul (Droplayer, outputweight), Outputbias) $  $ #label -Outputlabel = Tf.placeholder (Tf.float32, Shape=[none, 10]) -  the #loss Function and objective function -Loss = Tf.reduce_mean (Tf.nn.softmax_cross_entropy_with_logits_v2 (Labels=outputlabel, logits=outputlayer))Wuyitarget =Tf.train.AdamOptimizer (). Minimize (loss) the  - #record start training time WuStartTime =time.time () -  About #Training $ With TF. Session () as Sess: - Sess.run (Tf.global_variables_initializer ()) -BatchSize = 64 -      forIinchRange (1000): ABatch =Mnist.train.next_batch (batchsize) +Inputdata = Batch[0].reshape ([BatchSize, 28, 28, 1]) theLabeldata = batch[1] -Sess.run ([Target, loss], feed_dict={inputlayer:inputdata, outputlabel:labeldata}) $  thecorrected = Tf.equal (Tf.argmax (Outputlabel, 1), Tf.argmax (Outputlayer, 1)) theaccuracy =Tf.reduce_mean (tf.cast (corrected, tf.float32)) theAccuracyvalue = Sess.run (accuracy, feed_dict={inputlayer:inputdata, outputlabel:labeldata}) the         PrintI'Train Set accuracy:', Accuracyvalue) -  in     #Print End Time theEndTime =time.time () the     Print('Train Time:', EndTime-startTime) About  the     #Test thecorrected = Tf.equal (Tf.argmax (Outputlabel, 1), Tf.argmax (Outputlayer, 1)) theaccuracy =Tf.reduce_mean (tf.cast (corrected, tf.float32)) +Testimages = Mnist.test.images.reshape ([-1, 28, 28, 1]) -Testlabels =Mnist.test.labels theAccuracyvalue = Sess.run (accuracy, feed_dict={inputlayer:testimages, outputlabel:testlabels})Bayi     Print("accuracy on test set:", Accuracyvalue) the  theSess.close ()

View Code

This address: https://www.cnblogs.com/laishenghao/p/9738912.html

TensorFlow Training Mnist DataSet (3)--convolutional neural network