Convolutional Networks for Mnist in TensorFlow

It 's written in front .

This paper introduces the task of identifying handwritten characters by using convolution neural network based on TensorFlow on Mnist dataset, including: {Two layers of volume base}+{a layer of Relu full link layer}+{the full link layer of Softmax layer}. Because the structure is simple, the code is clear, the whole article to the main code, reading save effort and convenience. 1. Load mnist Data

# load Mnist data from
tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets (' Mnist_data ', one_hot=true)

2. Start TensorFlow interativesession

# start TensorFlow interativesession
import tensorflow as tf
sess = tf. InteractiveSession ()

3. Placeholders

# placeholders
x = Tf.placeholder (Tf.float32, Shape=[none, 784])
Y_ = Tf.placeholder (Tf.float32, Shape=[None, 10])

4. Weight initialization Function Define

# Weight Initialization
def weight_variable (shape):
    initial = Tf.truncated_normal (shape, stddev=0.1)
    Return TF. Variable (initial)
def bias_variable (shape):
    initial = Tf.constant (0.1, Shape=shape) return
    TF. Variable (initial)

5. Convolution and pooling Function Define

# convolution and pooling
def conv2d (X, W): Return
    tf.nn.conv2d (x, W, strides=[1, 1, 1, 1], padding= ' SAME ')

de F max_pool_2x2 (x): Return
    tf.nn.max_pool (x, Ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding= ' SAME ')

6. convolutional Layer

# convolutional Layer
w_conv1 = weight_variable ([5, 5, 1,])
b_conv1 = bias_variable ([+])
# (The Conv Olutional would compute features for each 5x5 patch)
# ([5, 5, 1, a], "5, 5": patch_size, 1:number of input Channe LS, 32:number of output Channels)

X_image = Tf.reshape (x, [-1, 1])
# (reshape x to a 4d tensor)
# (2n D & 3rd dimensions:image width and height. The final dimension:the number of color channels)

H_CONV1 = Tf.nn.relu (conv2d (X_image, W_CONV1) + b_conv1)
H_pool 1 = max_pool_2x2 (H_CONV1)

7. Sencond convolution Layer

# Sencond convolution Layer
w_conv2 = weight_variable ([5, 5,,,])
b_conv2 = bias_variable ([+]) H_

Conv2 = Tf.nn.relu (conv2d (h_pool1, w_conv2) + b_conv2)
h_pool2 = max_pool_2x2 (h_conv2)

8. Fully Connected Layer

# fully Connected Layer
w_fc1 = weight_variable ([7*7*64, 1024])
B_FC1 = bias_variable ([1024])

H_pool2_flat = Tf.reshape (H_pool2, [-1, 7*7*64])
H_FC1 = Tf.nn.relu (Tf.matmul (H_pool2_flat, W_FC1) + b_fc1)

9. Dropout

# dropout
Keep_prob = Tf.placeholder (tf.float32)
H_fc1_drop = Tf.nn.dropout (H_FC1, Keep_prob)

Readout Layer

# readout Layer
w_fc2 = weight_variable ([1024, ten])
B_FC2 = Bias_variable ([ten])

Y_conv = Tf.nn.softmax ( Tf.matmul (H_fc1_drop, W_FC2) + B_FC2)

Train and Evaluate the Model

# Train and Evaluate the Model cross_entropy = Tf.reduce_mean-tf.reduce_sum (y_ * Tf.log (Y_conv
), reduction_indices= [1]))
Train_step = Tf.train.AdamOptimizer (1e-4). Minimize (cross_entropy)
correct_prediction = tf.equal (Tf.argmax (y_ conv,1), Tf.argmax (y_,1))
accuracy = Tf.reduce_mean (Tf.cast (correct_prediction, Tf.float32))
Sess.run ( Tf.initialize_all_variables ()) for
I in range (20000):
  batch = Mnist.train.next_batch
  If i%100 = 0:
    train_accuracy = accuracy.eval (feed_dict={
        x:batch[0], Y_: batch[1], keep_prob:1.0})
    print ("Step%d, Training accuracy%g "% (I, train_accuracy))
  Train_step.run (feed_dict={x:batch[0], Y_: batch[1], keep_prob:0.5}) C12/>print ("Test accuracy%g"%accuracy.eval (feed_dict={
    x:mnist.test.images, Y_: Mnist.test.labels, Keep_prob: 1.0}))

The Output

/usr/bin/python2.7/home/zhao/pycharmprojects/tensorflow/tutorial/convolutional.py I tensorflow/stream_executor/ DSO_LOADER.CC:108] successfully opened CUDA Library libcublas.so locally I tensorflow/stream_executor/dso_loader.cc : 108] Successfully opened CUDA Library libcudnn.so locally I tensorflow/stream_executor/dso_loader.cc:108] successfully Opened CUDA Library libcufft.so locally I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA Library Libcuda.so.1 locally I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so
Locally extracting mnist_data/train-images-idx3-ubyte.gz extracting mnist_data/train-labels-idx1-ubyte.gz Extracting mnist_data/t10k-images-idx3-ubyte.gz extracting mnist_data/t10k-labels-idx1-ubyte.gz I tensorflow/core/ COMMON_RUNTIME/GPU/GPU_INIT.CC:102] Found device 0 with Properties:name:GeForce GTX 970 major:5 minor:2 Te (GHz) 1.266 pcibusid 0000:01:00.0 Total Memory:4.00gib free memory:3.59GiB I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] dma:0 i tensorflow/core/common_runtime/gpu/gpu_init.cc : 136] 0:y I tensorflow/core/common_runtime/gpu/gpu_device.cc:838] Creating TensorFlow device (/gpu:0)-> (device:0 , Name:geforce GTX 970, PCI bus id:0000:01:00.0) step 0, training accuracy 0.12 step, training accuracy 0.84 step 20 0, training accuracy 0.92 step, training accuracy 0.86 step, training accuracy 0.98 ... step 19700, training accur Acy 1 Step 19800, training accuracy 1 step 19900, training accuracy 0.98 W tensorflow/core/common_runtime/bfc_allocator.cc : 213] Ran out of the memory trying to allocate 2.59GiB. The caller indicates this is not a failure, but could mean that there could are performance gains if more memory is Avai
Lable. W tensorflow/core/common_runtime/bfc_allocator.cc:213] Ran out of the memory trying to allocate 1.34GiB. The caller indicates, this is not a failure, but could mean that there could are performance gains if more memoryis available. W tensorflow/core/common_runtime/bfc_allocator.cc:213] Ran out of the memory trying to allocate 3.90GiB. The caller indicates this is not a failure, but could mean that there could are performance gains if more memory is Avai
Lable. Test accuracy 0.9926

The test precision is up to 0.9926 , or it's good.