Learning notes Tf024:tensorflow implementing Softmax Regression (regression) recognition of handwritten numbers

TensorFlow implements Softmax Regression (regression) to recognize handwritten numbers. MNIST (Mixed National Institute of Standards and Technology database), Simple machine vision dataset, 28x28 pixel handwritten numerals, only grayscale value information, blank part 0, handwriting based on color depth [0 , 1], 784-dimensional, discard two-dimensional spatial information, the target is divided into 0~9 a total of 10 classes. Data load, data.read_data_sets, 55,000 samples, test set 10000 samples, validation set 5000 samples. Sample callout information, label,10 dimension vector, 10 kinds of one-hot encoding. Training set training model, validation set test results, test set evaluation model (accuracy rate, recall, f1-score).

Algorithm design, Softmax regression training handwritten numeral recognition classification model, estimating category probability, taking probability maximum number as model output result. Class features are added to determine class probabilities. Model learning training to adjust the weight value. Softmax, the various features calculate the EXP function, normalized (the output probability value for all categories is 1). y = Softmax (wx+b).

NumPy uses C, FORTRAN, call Openblas, MKL matrix Operations Library. TensorFlow dense complex operations are performed outside of Python. To define a calculation diagram, the operation does not need to pass the finished data back to Python every time, all running outside of Python.

Import tensor flow as TF, loaded into the TensorFlow library. less = TF. InteractiveSession (), create interactivesession, register as default session. Different session data, operations, independent of each other. x = Tf.placeholder (Tf.float32, [none,784]), create placeholder receive input data, first parameter data type, second parameter represents tensor shape data size. None is an unlimited number of inputs, each input is a 784-dimensional vector.

Tensor stores data and disappears once it is used. Variable is persisted in model training iterations, is long-lived, and is updated every iteration of the cycle. The variable object weights and biases of the Softmax regression model are initialized to 0. Model training automatically learns the appropriate values. Complex networks, initialization methods are important. w = tf. Variable (Tf.zeros ([784, 10]), 784 feature dimension, 10 class. Label,one-hot encoded 10-dimensional vector.

Softmax regression algorithm, y = Tf.nn.softmax (Tf.matmul (x, W) + b). The tf.nn contains a large number of neural network components. Tf.matmul, matrix multiplication function. TensorFlow will forward, backward content automatic implementation, as long as the definition of loss, training automatic derivation gradient decline, complete Softmax regression model parameters automatic learning.

Define the loss function description problem model classification accuracy. The smaller the loss, the smaller the result of the model classification and the real value, the more accurate. The initial parameters of the model are all zero, resulting in the initial loss. The training objective is to reduce the loss and find the global optimal or local optimal solution. Cross-entropy, classification problems commonly used loss function. Y predict the probability distribution, y ' true probability distribution (Label one-hot coding), judge the model to the true probability distribution prediction accuracy. Cross_entropy = Tf.reduce_mean (-tf.reduce_sum (Y_ * tf.log (y), reduction_indices=[1])). Define placeholder and enter the real label. Tf.reduce_sum sum, tf.reduce_mean each batch data result to mean value.

Define the optimization algorithm, the random gradient drops sgd (Stochastic Gradient descent). According to the calculation diagram automatic derivation, according to the backward propagation (back propagation) algorithm training, each round iteration update parameter reduces loss. Provides the package optimizer, each iteration feed data, TensorFlow in the background auto-replenishment operation (operation) to achieve reverse propagation and gradient descent. Train_step = Tf.train.GradientDescentOptimizer (0.5). Minimize (Cross_entropy). Call Tf.train.GradientDescentOptimizer, set the learning speed 0.5, set the optimization target cross-entropy, get the training operation Train_step.

Tf.global_variables_initializer (). Run (). TensorFlow Global parameter initializer Tf.golbal_variables_initializer.

Batch_xs,batch_ys = Mnist.train.next_batch (100). Training Operation Train_step. Each time randomly extracts 100 samples from the training set to form Mini-batch,feed to placeholder, and calls the Train_step training sample. With a small sample training, the random gradient drops and the convergence rate is faster. Each training sample, the calculation of a large amount, not easy to jump out of the local optimal.

Correct_prediction = Tf.equal (Tf.argmax (y,1), TF.ARGMZX (y_,1)), verifies the model accuracy rate. Tf.argmax from tensor to find the maximum sequence number, Tf.argmax (y,1) to predict the probability of the largest, Tf.argmax (y_,1) Find samples of the real number category. Tf.equal determines whether the predicted number category is correct, and returns whether the calculation classification operation is correct.

accuracy = Tf.reduce_mean (Tf.cast (Correct_prediction,tf.float32)), all samples are counted to predict the accuracy. Tf.cast conversion correct_prediction output value type.

Print (Accuracy.eval ({x:mnist.test.images,y_: Mnist.test.labels})). Test data characteristics, label input evaluation process, calculate model test set accuracy rate. Softmax Regression mnist Data Classification and recognition, the average accuracy rate of test set is about 92%.

TensorFlow implement simple machine algorithm steps:
1､ defines the algorithm formula, the neural network forward calculates.
2､ defines the loss, selects the optimizer, and specifies the optimizer to optimize the loss.
3､ Iterative training data.
4､ test set, validation set evaluation accuracy rate.

The definition formula is just computation Graph, only call the Run method, feed data, calculation is executed.

     fromTensorflow.examples.tutorials.mnistImportInput_data mnist= Input_data.read_data_sets ("mnist_data/", one_hot=True)Print(Mnist.train.images.shape, Mnist.train.labels.shape)Print(Mnist.test.images.shape, Mnist.test.labels.shape)Print(Mnist.validation.images.shape, Mnist.validation.labels.shape)ImportTensorFlow as TF sess=TF. InteractiveSession () x= Tf.placeholder (Tf.float32, [None, 784]) W= TF. Variable (Tf.zeros ([784, 10])) b= TF. Variable (Tf.zeros ([10])) y= Tf.nn.softmax (Tf.matmul (x, W) +b) Y_= Tf.placeholder (Tf.float32, [None, 10]) cross_entropy= Tf.reduce_mean (-tf.reduce_sum (Y_ * tf.log (y), reduction_indices=[1])) Train_step= Tf.train.GradientDescentOptimizer (0.5). Minimize (Cross_entropy) Tf.global_variables_initializer (). Run () forIinchRange (1000): Batch_xs, Batch_ys= Mnist.train.next_batch (100) Train_step.run ({x:batch_xs, y_: Batch_ys}) correct_prediction= Tf.equal (Tf.argmax (y, 1), Tf.argmax (Y_, 1)) Accuracy=Tf.reduce_mean (Tf.cast (correct_prediction, tf.float32))Print(Accuracy.eval ({x:mnist.test.images, y_: Mnist.test.labels}))

Resources:
"TensorFlow Practice"

Welcome to pay consultation (150 yuan per hour), my: Qingxingfengzi