Learning notes TF028: simple convolution network and learning notes tf028 convolution

Learning notes TF028: simple convolution network and learning notes tf028 convolution

Load the MNIST dataset. Create the default Interactive Session.

The initialization function creates random noise to break the complete symmetry. Truncates Normal Distribution noise, with a standard deviation of 0.1. ReLU, offset plus a small positive value (0.1) to avoid dead nodes (dead neurons ).

Convolution function, tf. nn. conv2d, TensorFlow 2-dimensional convolution function, parameter x Input, W convolution parameter, convolution kernel size, number of channels, and number of convolution kernels (number of features extracted from the convolution Layer ). The moving step of the Strides convolution template. "All 1" means no omission across each vertex of the image. Padding indicates the boundary processing method. The SAME boundary is added with Padding, And the convolution output and input are kept in the SAME size.

Pooling layer functions, tf. nn. max_pool, and maximum pooling functions of TensorFlow. 2x2 maximum pooling, 2x2 pixel block down to 1x1 pixel. The largest pooling retains the highest gray level of the original pixel block and the most significant features. Strides has two steps in the horizontal and vertical directions.

Defines the input placeholder, x feature, and y real label. Convolutional Neural Network spatial structure information, 1D input vector, converted to 2D image structure. Dimensions: [-,]. -1 indicates that the number of samples is not fixed. 1 indicates the number of color channels. Tf. reshape tensor deformation function.

The first convolution layer, convolution function initialization, weights, bias. [5, 5,] indicates the convolution kernel size of 5x5, one color channel, and 32 different convolution kernels. Conv2d functions are used for Convolution, bias is added, and ReLU is used to activate non-linear processing of functions. Use the max_pool_2x2 pooling function to perform convolution output.

The second convolutional layer has 64 convolution kernels and 64 features are extracted. The two step sizes are 2x2 in the largest pool, with the side length of only 1/4, and the image size is 28x28 to 7x7. The number of convolution cores in the second convolution layer is 64, and the output tensor size is 7x7x64. Use the tf. reshape function to output tensor deformation to the second convolutional layer, convert it to a 1D vector, connect to a fully connected layer, and use the hidden node 1024 to activate the function using the ReLU function.

Use the Dropout layer to reduce overfitting. Dropout, pass in keep_prob ratio control through a placeholder. During training, some node data is randomly discarded to reduce overfitting, and all data is retained for the best prediction performance.

The Dropout layer output connects to the Softmax layer to obtain the final probability output.

Define the loss function cross_entropy. The optimizer uses Adam to give a small learning rate of 1e-4.

Define evaluation accuracy operations.

Train, Initialize all parameters, set the keep_prob ratio of Dropout during training to 0. 5. Use the mini-batch with a size of 50 for 20000 training iterations with a sample count of 1 million. Every 100 training sessions, evaluation accuracy, keep_prob set 1, real-time monitoring of model performance.

All training is completed, and the test set is fully tested to obtain the overall classification accuracy.

99.2% accuracy, convolutional network abstracts image features, and shares convolution core weights.

    from tensorflow.examples.tutorials.mnist import input_data    import tensorflow as tf    mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)    sess = tf.InteractiveSession()    def weight_variable(shape):      initial = tf.truncated_normal(shape, stddev=0.1)      return tf.Variable(initial)    def bias_variable(shape):      initial = tf.constant(0.1, shape=shape)      return tf.Variable(initial)      def conv2d(x, W):      return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')    def max_pool_2x2(x):      return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],                        strides=[1, 2, 2, 1], padding='SAME')                              x = tf.placeholder(tf.float32, [None, 784])    y_ = tf.placeholder(tf.float32, [None, 10])    x_image = tf.reshape(x, [-1,28,28,1])                            W_conv1 = weight_variable([5, 5, 1, 32])    b_conv1 = bias_variable([32])    h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)    h_pool1 = max_pool_2x2(h_conv1)    W_conv2 = weight_variable([5, 5, 32, 64])    b_conv2 = bias_variable([64])    h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)    h_pool2 = max_pool_2x2(h_conv2)    W_fc1 = weight_variable([7 * 7 * 64, 1024])    b_fc1 = bias_variable([1024])    h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])    h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)    keep_prob = tf.placeholder(tf.float32)    h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)    W_fc2 = weight_variable([1024, 10])    b_fc2 = bias_variable([10])    y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)    cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))    train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)    correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))    tf.global_variables_initializer().run()    for i in range(20000):      batch = mnist.train.next_batch(50)      if i%100 == 0:        train_accuracy = accuracy.eval(feed_dict={        x:batch[0], y_: batch[1], keep_prob: 1.0})        print("step %d, training accuracy %g"%(i, train_accuracy))      train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})    print("test accuracy %g"%accuracy.eval(feed_dict={    x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))

 

References:
TensorFlow practice

Welcome to paid consultation (150 RMB per hour), My: qingxingfengzi