TensorFlow實戰——CNN（LeNet5）——MNIST數字識別

本文地址：
http://blog.csdn.net/u011239443/article/details/72861591

我們來實現下不標準的LeNet模型：
train：https://github.com/xiaoyesoso/TensorFlowinAction/blob/master/InActionB1/chapter6/mnist_train_6_4_1.py
inference：https://github.com/xiaoyesoso/TensorFlowinAction/blob/master/InActionB1/chapter6/mnist_inference_6_4_1.py train

train部分和《TensorFlow實戰——DNN——MNIST數字識別 》中沒太多的不同。 首先，原始學習率要降低：

LEARNING_RATE_BASE = 0.01

第二點，x是一個四維的矩陣：

x = tf.placeholder(tf.float32,[BATCH_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.NUM_CHANNELS],name='x-input')

mnist_inference_6_4_1.NUM_CHANNELS為圖片的深度。 xs也要換成四維矩陣：

            xs,ys = mnist.train.next_batch(BATCH_SIZE)            reshaped_xs = np.reshape(xs,(BATCH_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.NUM_CHANNELS))            _,loss_value,step = sess.run([train_op,loss,global_step],feed_dict={x:reshaped_xs,y_:ys})

inference layer1

    with tf.variable_scope('layer1-conv1'):        conv1_weights = tf.get_variable("weight",[CONV1_SIZE,CONV1_SIZE,NUM_CHANNELS,CONV1_DEEP],initializer=tf.truncated_normal_initializer(stddev=0.1))        conv1_biases = tf.get_variable("bias",[CONV1_DEEP],initializer=tf.constant_initializer(0.0))        conv1 = tf.nn.conv2d(input_tensor,conv1_weights,strides=[1,1,1,1],padding='SAME')        relu1 = tf.nn.relu(tf.nn.bias_add(conv1,conv1_biases))

首先我們看下strides參數：

strides: A list of ints.
1-D of length 4. The stride of the sliding window for each dimension
of input. Must be in the same order as the dimension specified with format.

strides代表著移動的步長，它的順序必須和input_tensor一樣，及[BATCH_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.NUM_CHANNELS]。由於BATCH_SIZE和mnist_inference_6_4_1.NUM_CHANNELS上肯定是一步一步的移動的，所以數組的第一個值和最後一個值肯定為1。 padding='SAME'，表示填充0，不改變Image的大小。 注意tf.nn.bias_add(conv1,conv1_biases)，並不是conv1與conv1_biases直接相加。 layer2

    with tf.name_scope('layer2-pool1'):        pool1 = tf.nn.max_pool(relu1,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')

max_pool表示是取最大值的池化層。
我們來看下參數ksize：

ksize: A list of ints that has length >= 4. The size of the window for
each dimension of the input tensor.

視窗各個維度多大小。由於池化層視窗只在當前資料中的當前深度做，所以數組的第一個值和最後一個值肯定為1。 layer5

layer3和layer4前面的類似，我們跳過它們來看layer5：

    pool_shape = pool2.get_shape().as_list()    nodes = pool_shape[1] * pool_shape[2] * pool_shape[3]    reshaped = tf.reshape(pool2,[pool_shape[0],nodes])    with tf.variable_scope('layer5-fc1'):        fc1_weights = tf.get_variable("weight", [nodes,FC_SIZE],                                        initializer=tf.truncated_normal_initializer(stddev=0.1))        if regularizer != None:            tf.add_to_collection('losses',regularizer(fc1_weights))        fc1_biases = tf.get_variable("bias", [FC_SIZE], initializer=tf.constant_initializer(0.1))        fc1 = tf.nn.relu(tf.matmul(reshaped,fc1_weights) + fc1_biases)        if train:            fc1 = tf.nn.dropout(fc1,0.5)

get_shape().as_list()能得到pool2的size。
pool_shape[1]∗pool_shape[2]∗pool_shape[3]=長×寬×深 pool\_shape[1] * pool\_shape[2] * pool\_shape[3] = 長×寬×深，這相當把長方體拉成一條直線。pool_shape[0]為BATCH_SIZE dropout為了賦值過擬合，可以似的一定比例的輸出變為0。

其餘部分就是全串連神經網路了，layer6也和layer5類似。

結果：

After 1 training step(s), loss is 6.06818After 101 training step(s), loss is 2.24668After 201 training step(s), loss is 1.65929After 301 training step(s), loss is 1.30799After 401 training step(s), loss is 1.3613After 501 training step(s), loss is 0.960646After 601 training step(s), loss is 0.954722After 701 training step(s), loss is 0.883449After 801 training step(s), loss is 0.870421After 901 training step(s), loss is 0.905906After 1001 training step(s), loss is 0.932337