TensorFlow QuickStart 2--enabling handwritten digit recognition

TensorFlow Quick start for handwritten digit recognition

Environment: Virtual machine ubuntun16.0.4 TensorFlow (CPU version only)
TensorFlow installation See:
http://blog.csdn.net/yhhyhhyhhyhh/article/details/54429034
Or:
Http://www.tensorfly.cn/tfdoc/get_started/os_setup.html

In this paper, we will use TensorFlow to test mnist data set with Softmax regression and convolutional neural network, and quickly realize the test of handwritten numeral recognition. Only TensorFlow as an exercise, do not explain too many models, frameworks and other theoretical knowledge.

Directory

1.MNIST DataSet 2.softmax Regression test MNIST3. convolutional Neural Network test Mnist4.tensorflow learning document 1.MNIST datasets

Mnist Data Set official website: http://yann.lecun.com/exdb/mnist/
Follow the TensorFlow Chinese community tutorial to load mnist data training data and test data into your own code.
1) Download Mnist data set
input_data.py File Download: input_data.py
Running in terminal: Python ~/test/input_data.py
(My datasets and input_data.py files are placed under the test folder in the home directory, note the path.) ）
2) load into the code
Add the following to the Python code that uses the Mnist data:

#导入MNIST数据import input_datamnist = input_data.read_data_sets(‘MNIST_data/‘, one_hot=True)

Each mnist data unit consists of two parts (both the training dataset and the test data set ): A picture with handwritten numbers and a corresponding label.
Therefore, in the Mnist training dataset, Mnist.train.images is a tensor of shape [60000, 784], the first dimension number is used to index the picture, and the second dimension number is used to index the pixels in each picture. Each element in this tensor represents the intensity value of a pixel in a picture, with values between 0 and 1.

The corresponding mnist dataset is labeled with a number from 0 to 9, which describes the number represented in a given picture. In order to use this tutorial, we make the label data "one-hot vectors". A one-hot vector in addition to the number of one digit is 1 other than the number of dimensions is 0. So in this tutorial, the number n is represented as a 10-dimensional vector that has a number of 1 only in the nth dimension (starting at 0). For example, label 0 will be represented as ([1,0,0,0,0,0,0,0,0,0,0]). Therefore, Mnist.train.labels is a digital matrix of [60000, 10].

2.softmax regression test mnist

1) Softmax regression model
Click to view a post in detail on Softmax regression:

Simply put, Softmax regression is the generalization of logistic regression to multi-classification problem, when it is two classification, it is logistic regression. Softmax regression is more suitable for mutual exclusion among categories, such as character recognition. The Softmax cost function and the logistic cost function are very similar in form, except that the possible values of the class tag are accumulated in the Softmax loss function.

2) Simple test mnist
Note the path.

yhh@ubuntu:~/Test$ python input_data.pyyhh@ubuntu:~/Test$ python softmax_tf_mnist.py

Complete code for the Softmax regression test mnist:
softmax_tf_mnist.py

#-*-Coding:utf-8-*-" "Softmax regression test mnist data set" "Import Input_dataimport TensorFlow as TF#导入MNIST数据Mnist = Input_data. Read_data_sets ("mnist_data/", one_hot=true)#softmax回归模型中的x, W,bx= TF. Placeholder("Float", [None,784]) W = TF. Variable(TF. Zeros([784,Ten])) B = tf. Variable(TF. Zeros([Ten]))#softmax回归模型y= TF. NN. Softmax(TF. Matmul(x, W) + b) Y_ = TF. Placeholder("Float", [None,Ten])#计算交叉熵Cross_entropy =-TF. Reduce_sum (Y_*TF. Log(y))#设置TensorFlow用梯度下降算法以0.01 Learning Rate Minimization Crossover entropyTrain_step = TF. Train. Gradientdescentoptimizer(0.01). Minimize(cross_entropy)#初始化变量init = TF. Initialize_all_variables ()#评估模型Correct_prediction = TF. Equal(TF. Argmax(y,1), TF. Argmax(Y_,1)) accuracy = TF. Reduce_mean (TF. Cast(Correct_prediction,"Float"))#开启TesnsorflowSess = TF. Session() Sess. Run(init)#循环训练模型For IinchRange +): Batch = Mnist. Train. Next_batch ( -) Sess. Run(train_step,feed_dict={x: batch[0], Y_: batch[1]})#输出结果Print"Softmax regression test mnist Data Set accuracy:"Print Sess. Run(Accuracy, feed_dict={x: mnist. Test. Images, Y_: Mnist. Test. Labels})

As a result, in order to reduce the running time, I directly set the number of training is very small.
When the number of workouts is set to 1000:

3. convolutional Neural Network Test mnist

1) convolutional Neural network
A blog post explaining CNN:
http://www.36dsj.com/archives/24006

2) Test Mnist

yhh@ubuntu:~/Test$ python cnn_tf_mnist.py

If the mnist dataset is not imported, run it first:

yhh@ubuntu:~/Test$ python input_data.py

As a result, in order to reduce the running time, I directly set the number of training is very small.

When the number of workouts is set to 1000:

convolutional Neural Network Test Mnist's complete code:
cnn_tf_mnist.py

#-*-Coding:utf-8-*-" convolutional neural network test mnist data"#导入MNIST数据ImportInput_dataImportTensorFlow asTfmnist = Input_data.read_data_sets (' mnist_data/', one_hot=True) Sess = tf. InteractiveSession () x = Tf.placeholder ("Float", shape=[None,784]) Y_ = Tf.placeholder ("Float", shape=[None,Ten]) W = tf. Variable (Tf.zeros ([784,Ten])) b = tf. Variable (Tf.zeros ([Ten])) Sess.run (Tf.initialize_all_variables ())#权重初始化函数, initialize the offset with a small positive number def weight_variable(Shape):Initial = Tf.truncated_normal (Shape, stddev=0.1)returnTf. Variable (initial) def bias_variable(Shape):Initial = Tf.constant (0.1, Shape=shape)returnTf. Variable (initial)#卷积和池化函数 def conv2d(x, W):  returntf.nn.conv2d (x, W, strides=[1,1,1,1], padding=' Same ') def max_pool_2x2(x):  returnTf.nn.max_pool (x, ksize=[1,2,2,1], strides=[1,2,2,1], padding=' Same ')#第一层卷积W_CONV1 = Weight_variable ([5,5,1, +]) B_conv1 = Bias_variable ([ +])#把x变成一个4d向量X_image = Tf.reshape (x, [-1, -, -,1])#把x_image和权值向量进行卷积, plus the offset, then apply the Relu activation function,#进行池化. H_CONV1 = Tf.nn.relu (conv2d (X_image, W_CONV1) + b_conv1) H_pool1 = max_pool_2x2 (H_CONV1)#第二层卷积W_conv2 = Weight_variable ([5,5, +, -]) B_conv2 = Bias_variable ([ -]) H_conv2 = Tf.nn.relu (conv2d (h_pool1, w_conv2) + b_conv2) H_pool2 = max_pool_2x2 (h_conv2)#密集连接层W_FC1 = Weight_variable ([7*7* -,1024x768]) B_FC1 = Bias_variable ([1024x768]) H_pool2_flat = Tf.reshape (H_pool2, [-1,7*7* -]) H_fc1 = Tf.nn.relu (Tf.matmul (H_pool2_flat, W_FC1) + b_fc1)#为了减少过拟合, add dropout before the output layerKeep_prob = Tf.placeholder ("Float") H_fc1_drop = Tf.nn.dropout (H_FC1, Keep_prob)#添加一个softmax层, just like Softmax regression.W_FC2 = Weight_variable ([1024x768,Ten]) B_FC2 = Bias_variable ([Ten]) Y_conv=tf.nn.softmax (Tf.matmul (H_fc1_drop, W_FC2) + B_FC2)#训练设置Cross_entropy =-tf.reduce_sum (Y_*tf.log (y_conv)) Train_step = Tf.train.AdamOptimizer (1e-4). Minimize (cross_entropy) correct_prediction = Tf.equal (Tf.argmax,1), Tf.argmax (Y_,1)) accuracy = Tf.reduce_mean (Tf.cast (Correct_prediction,"Float")) Sess.run (Tf.initialize_all_variables ())#训练 forIinchRange +): Batch = Mnist.train.next_batch ( -)ifi% -==0: Train_accuracy = Accuracy.eval (feed_dict={x:batch[0], Y_: batch[1], Keep_prob:1.0})Print "-->step%d, training accuracy%.4f"% (i, train_accuracy) Train_step.run (feed_dict={x:batch[0], Y_: batch[1], Keep_prob:2.W})#最终评估Print "convolutional neural network test mnist Data Set accuracy:%g"%accuracy.eval (feed_dict={x:mnist.test.images, Y_: Mnist.test.labels, Keep_prob:1.0})

4.Tensorflow Learning Documentation

Tenssorflow Chinese Community:
Http://www.tensorfly.cn/tfdoc/tutorials/mnist_pros.html

TensorFlow QuickStart 2--enabling handwritten digit recognition