TensorFlow starting from 0 (4)--Interpreting Mnist Program _ Machine Learning

Objective

Because of the problem of image Learning machine learning, choose TensorFlow, but seems to go directly from the example of imagenet, but found how to find the end (Python will not, machine learning also do not understand), but according to my past experience, in this situation, and no discerning to the road, when the blind touch (yes , is a blind touch), starting from one o'clock, what to see, look at the relevant knowledge, it is best to repeat more than two times, and then grabbed one of the examples, contact the things that have been seen before, deepen the understanding.
Of course, if only the ready-made things to use, it is not so complicated, a simple understanding of the line. But to be flexible, the above is my method. Background information

TensorFlow official online Several examples of tutorials, simple to look at is not difficult (the individual is a machine learning Zero Foundation, math is not what kind of person, so it seems really easy), as long as the link provided above, read some relevant paper, the following is a link to the official website tutorial.
Https://www.tensorflow.org/versions/r0.9/tutorials/index.html
I wanted to start with imagenet, but it did not teach the model how to build, directly to a model file, loaded in. So do not go back and start with the simplest example. This is the mnist (handwriting recognition) tutorial. Mnist

This is a thing, everyone Google.
TensorFlow's official website gives two examples, simple examples, through the General machine learning algorithm to achieve. The concept involved in the Last post mentioned, you can see for yourselves: Logistic regression relu activation activation function dropout Softmax regression cross cross entropy entropy function cost Function/loss function gradient descent gradient descent

Personally feel not fully understand also does not matter, strewn, there is always a process, I do not understand ...

There is also an example of CNN, which is my purpose. Realization of Mnist Neural network

According to the official website that this recognition rate is higher, I believe, because this is my purpose, the following is the code, my machine storage path  
~/libsource/tensorflow/tensorflow/models/image/ Mnist:

# Copyright 2015 the TensorFlow Authors.
All Rights Reserved.
# # Licensed under the Apache License, Version 2.0 (the "License");
# You could not use this file, except in compliance with the License. # You may obtain a copy of the License in # # http://www.apache.org/licenses/LICENSE-2.0 # unless required by applic Able or agreed to in writing, software # Distributed under the License be distributed on ' as is ' basis, # without W
Arranties or CONDITIONS of any KIND, either express OR implied.
# The License for the specific language governing permissions and # Limitations under the License. # ============================================================================== "" "Simple, End-to-end,

Lenet-5-like convolutional mnist Model example. This should achieve a test error of 0.7%.
Please keep this model as simple and linear as possible, it are meant as a tutorial for simple convolutional models.
Run with--self_test on the command line to execute a short self-test. "" "From__future__ Import absolute_import from __future__ Import division from __future__ import print_function import gzip Impor T OS import sys import time import numpy to six.moves import urllib from six.moves import xrange # pylint:disable=red Efined-builtin import TensorFlow as tf source_url = ' http://yann.lecun.com/exdb/mnist/' work_directory = ' data ' Image_siz
E = Num_channels = 1 Pixel_depth = 255 Num_labels = Ten validation_size = 5000 # SIZE of the VALIDATION set.
Seed = 66478 # Set to None for random seed.


Batch_size = Num_epochs = Eval_batch_size = Eval_frequency = + # Number of steps between.
Tf.app.flags.DEFINE_boolean ("Self_test", False, "True if running a self test.")  FLAGS = Tf.app.flags.FLAGS def maybe_download (filename): "" "Download the data from Yann ' s website, unless it ' s already
  here. "" " If not tf.gfile.Exists (work_directory): Tf.gfile.MakeDirs (work_directory) filepath = Os.path.join (Work_directory, fi Lename) If not tf.gfile. Exists (filepath): filepath, _ = Urllib.request.urlretrieve (Source_url + filename, filepath) with Tf.gfile.GFile (fi
  Lepath) as F:size = F.size () print (' successfully downloaded ', filename, size, ' bytes. ') Return filepath def extract_data (filename, num_images): "" "extract the images into a 4D tensor [image index, y, x, Cha

  Nnels].
  Values are rescaled from [0, 255] down to [-0.5, 0.5]. "" "Print (' extracting ', filename) with gzip.open (filename) as bytestream:bytestream.read buf = ByteStream
    . Read (Image_size * image_size * num_images) data = Numpy.frombuffer (buf, dtype=numpy.uint8). Astype (Numpy.float32) data = (data-(pixel_depth/2.0))/pixel_depth data = Data.reshape (Num_images, Image_size, image_size, 1) retur
  n Data def extract_labels (filename, num_images): "" "extract the labels into a vector of int64 label IDs." " Print (' extracting ', filename) with gzip.open (filename) as Bytestream:bytestream.read (8) buf = bYtestream.read (1 * num_images) labels = Numpy.frombuffer (buf, dtype=numpy.uint8). Astype (Numpy.int64) return labels
  def fake_data (num_images): "" "Generate a fake dataset that matches the dimensions of mnist." " data = Numpy.ndarray (shape= (Num_images, Image_size, Image_size, num_channels), Dtype=numpy.float32) labels = Numpy.zeros (Shape= (Num_images,), Dtype=numpy.int64) for image in Xrange (num_images): label = image% 2 Data[im Age,:,:, 0] = label-0.5 labels[image] = label return data, Labels def error_rate (predictions, labels): "" "Re
  Turn the error rate based on dense predictions and sparse labels. "" "  Return 100.0-(100.0 * numpy.sum (Numpy.argmax (predictions, 1) = labels)/predictions.shape[0]) def
    Main (Argv=none): # pylint:disable=unused-argument if FLAGS.self_test:print (' Running self-test. ') Train_data, Train_labels = Fake_data (256) validation_data, validation_labels = Fake_data (EVAL_BATCH_SIZE) Test_data, test_labels = Fake_data (eval_batch_size) Num_epochs = 1 Else: # get the data. Train_data_filename = Maybe_download (' train-images-idx3-ubyte.gz ') Train_labels_filename = Maybe_download ('  Train-labels-idx1-ubyte.gz ') Test_data_filename = maybe_download (' t10k-images-idx3-ubyte.gz ') test_labels_filename
    = Maybe_download (' t10k-labels-idx1-ubyte.gz ') # Extract it into numpy arrays.
    Train_data = Extract_data (Train_data_filename, 60000) Train_labels = Extract_labels (Train_labels_filename, 60000) Test_data = Extract_data (Test_data_filename, 10000) Test_labels = Extract_labels (Test_labels_filename, 10000) # G
    Enerate a validation set.
    Validation_data = Train_data[:validation_size, ...]
    Validation_labels = train_labels[:validation_size] Train_data = train_data[validation_size:, ...] Train_labels = train_labels[validation_size:] Num_epochs = Num_epochs train_size = train_labels.shape[0] # this I s where TraiNing samples and labels are fed to the graph.  # These placeholder nodes would be fed a batch of training data at each # training step using the ' {feed_dict} argument to
  The Run () call below. Train_data_node = Tf.placeholder (Tf.float32, shape= (Batch_size, Image_size, image_size)) NUM_CHANNELS N_labels_node = Tf.placeholder (Tf.int64, shape= (Batch_size,)) Eval_data = Tf.placeholder (Tf.float32, shape=

  (Eval_batch_size, Image_size, Image_size, Num_channels)) # The variables below hold all the trainable weights. They are passed an # initial value which would be assigned when we call: # {Tf.initialize_all_variables (). Run ()} conv 1_weights = tf.
                          Variable ([5, 5, Num_channels, Tf.truncated_normal], # 5x5 filter, Depth 32. stddev=0.1, seed=seed)) conv1_biases = tf. Variable (tf.zeros)) conv2_weights = tf. Variable (Tf.truncated_normal (5, 5,,), STDdev=0.1, seed=seed)) conv2_biases = tf. Variable (Tf.constant (0.1, shape=[64)) fc1_weights = tf.
      Variable (# Fully connected, depth 512.
  Tf.truncated_normal ([image_size//4 * image_size//4 *), stddev=0.1, Seed=seed)) fc1_biases = tf. Variable (Tf.constant (0.1, shape=[512)) fc2_weights = tf. Variable (Tf.truncated_normal, Num_labels), stddev=0.1, d=seed)) fc2_biases = tf. Variable (Tf.constant (0.1, Shape=[num_labels)) # We'll replicate the model structure for the training subgraph, as We
  LL # as the evaluation subgraphs while sharing the trainable parameters.
    DEF model (data, Train=false): "" "" "" the Model definition. "" # 2D convolution, with ' SAME ' padding (i.e. the output feature map has # The SAME size as the input).
   Note This {strides} is a 4D array whose # shape matches the data layout: [image index, y, x, depth]. CONV = tf.nn.conv2d (data, Conv1_weights, strides=[1, 1, 1, 1],
    padding= ' SAME ') # Bias and rectified linear non-linearity. Relu = Tf.nn.relu (Tf.nn.bias_add (conv, conv1_biases)) # Max pooling. The kernel size spec {ksize} also follows the layout of # the data.
    Here we have a pooling window of 2, and a stride of 2.
                          Pool = Tf.nn.max_pool (Relu, Ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
                        padding= ' SAME ') conv = tf.nn.conv2d (pool, conv2_weights, Strides=[1, 1, 1, 1], padding= ' SAME ') Relu = Tf.nn.relu (Tf.nn.bias_add (CONV, conv 2_biases)) pool = Tf.nn.max_pool (Relu, Ksize=[1, 2, 2, 1], strides =[1, 2, 2, 1], padding= ' SAME ') # reshape the feature map cuboid into a 2D mAtrix to feeds it to the # fully connected layers. Pool_shape = Pool.get_shape (). As_list () reshape = Tf.reshape (pool, [pool_shape[0], pool_shape[1] * PO OL_SHAPE[2] * pool_shape[3]) # fully connected layer.
    Note the ' + ' operation automatically # broadcasts the biases. Hidden = Tf.nn.relu (Tf.matmul (reshape, fc1_weights) + fc1_biases) # Add A, 50% dropout during only.
    Dropout also scales # activations such that no rescaling was needed at evaluation time.  If Train:hidden = Tf.nn.dropout (hidden, 0.5, seed=seed) return Tf.matmul (hidden, fc2_weights) + fc2_biases
  Training computation:logits + cross-entropy loss. Logits = Model (Train_data_node, True) loss = Tf.reduce_mean (Tf.nn.sparse_softmax_cross_entropy_with_logits (logits
  , Train_labels_node)) # L2 regularization for the fully connected parameters. Regularizers = (Tf.nn.l2_loss (fc1_weights) + Tf.nn.l2_loss (fc1_biases) + TF.NN.L2_loss (fc2_weights) + Tf.nn.l2_loss (fc2_biases)) # ADD The regularization term to the loss. Loss + 5e-4 * regularizers # Optimizer:set up a variable that ' s incremented once/batch and # controls the Learn
  ing rate decay. Batch = tf.
  Variable (0) # decay once per epoch, using a exponential schedule starting at 0.01.
      Learning_rate = Tf.train.exponential_decay (0.01, # Base learning rate.
      Batch * batch_size, # Current index into the dataset.
      Train_size, # decay step.
      0.95, # decay rate.
  Staircase=true) # Use simple momentum for the optimization.
                                                       Optimizer = Tf.train.MomentumOptimizer (learning_rate, 0.9). Minimize (loss,
  Global_step=batch) # Predictions for the current training minibatch. Train_prediction = Tf.nn.softmax (logits) # Predictions for the test and validation, which we ' ll compute lessEn. Eval_prediction = Tf.nn.softmax (model (Eval_data)) # Small utility function to evaluate a dataset by feeding batches of
  Data to # {Eval_data} and pulling the results from {eval_predictions}.
  # Saves memory and enables this to run on smaller GPUs. Size = data.shape[0] If size < Eval_batch_size:raise valueerror ("BATCH size for evals larger than dataset:%  D "% size) predictions = Numpy.ndarray (shape= (Size, num_labels), dtype=numpy.float32) for the begin in Xrange (0, size, Eval_batch_size): end = begin + Eval_batch_size If End <= size:predictions[begin:end,:] = SESS.R
      Un (eval_prediction, Feed_dict={eval_data:data[begin:end, ...]}) Else:batch_predictions = Sess.run (eval_prediction, FEED_DICT={EVAL_DATA:DATA[-EVAL_BATC
        H_size:,...]} Predictions[begin:,:] = Batch_predictions[begin-size:,:] Return predictions # Create a local sessions to run the training. Start_time = Time.time () with TF.
    Session () as Sess: # Run All the initializers to prepare the trainable parameters.
    Tf.initialize_all_variables (). Run () print (' initialized! ')
    # Loop through training steps.  For step in xrange (int (NUM_EPOCHS * train_size)//Batch_size): # Compute The offset of the current minibatch in the
      Data.
      # We could use better randomization across epochs. 
      Offset = (step * batch_size)% (train_size-batch_size) Batch_data = Train_data[offset: (offset + batch_size), ...] Batch_labels = Train_labels[offset: (offset + batch_size)] # This dictionary maps the batch data (as a numpy a
      Rray) to the ' # node in the graph it should being fed to. Feed_dict = {train_data_node:batch_data, train_labels_node:batch_labels} # Run the graph and FE
Tch some of the nodes.      _, L, LR, predictions = Sess.run ([Optimizer, loss, Learning_rate, train_prediction], feed_dict= feed_dict) If step% eval_frequency = = 0:elapsed_time = Time.time ()-Start_time start_time = time.
               Time () print (' Step%d "(Epoch%.2f),%.1f ms '% (step, float (step) * Batch_size/train_size,
        1000 * elapsed_time/eval_frequency)) print (' Minibatch loss:%.3f, Learning rate:%.6f '% (l, LR)) Print (' Minibatch error:%.1f%% '% error_rate (predictions, batch_labels)) print (' Validation error:%.1f%% '% Erro R_rate (Eval_in_batches (Validation_data, Sess), validation_labels)) Sys.stdout.flush () # Finally P
    Rint the result!  Test_error = Error_rate (Eval_in_batches (Test_data, Sess), test_labels) print (' Test error:%.1f%% '% test_error) if FLAGS.self_test:print (' Test_error ', test_error) assert test_error = = 0.0, ' expected 0.0 test_error, got%.2f ' % (Test_error,) if __name__ = = ' __main__ ': Tf.app.run () Its TensorFlow figure is shown in the following illustration: 

Reading because I didn't write Python, I was puzzled at first, why does this product have two main function portals:

if __name__ = = ' __main__ ':

def Main (argv=none):  # Pylint:disable=unused-argument

Explain the following two posts:
Http://stackoverflow.com/questions/419163/what-does-if-name-main-do
Http://stackoverflow.com/questions/4041238/why-use-def-main

Cond....