Learn TensorFlow, save learning Network structure parameters and call

In deep learning, regardless of the learning framework, we encounter an important problem, that is, after training, how to store the depth of the network parameters. How these network parameters are invoked at the time of the test. In response to these two questions, this blog post explores how TensorFlow solves them. This blog is divided into three parts, the first is to explain tensorflow related functions, the second is the code routines, the third is the results of the operation.

A TensorFlow-related function

We say these two functions are mainly done by a class, the class Tf.train.Saver

[plain] view plain Copy saver = tf.train.Saver () Save_path = Saver.save (Sess, model_path) Load_path = SAVER.R eSTORE (Sess, model_path) saver = Tf.train.Saver () The class creates an object saver that is used to save and invoke the learned network parameters, which are stored in checkpoints

Save_path = Saver.save (Sess, Model_path) saves the learned network parameters to the Model_path path

Load_path = Saver.restore (Sess, Model_path) invokes the saved network parameters in the Model_path path into graph

Two code routines

[Python] View plain copy ' '   save and restore a model using tensorflow.  this  example is using the MNIST database of handwritten digits  ( http://yann.lecun.com/exdb/mnist/)     author: aymeric damien  project: https:// github.com/aymericdamien/tensorflow-examples/  ' '       # import minst  data   from tensorflow.examples.tutorials.mnist import input_data   Mnist  = input_data.read_data_sets ("/tmp/data/",  one_hot=true)       import  tensorflow as tf      # parameters   learning_rate = 0.001    batch_size = 100   display_step = 1   model_path =   "/home/lei/tensorflow-examples-master/examples/4_utils/model.ckpt"       # network  parameters   N_hidden_1 = 256 # 1st layer number of features    n_hidden_2 = 256 # 2nd layer number of features    n_input = 784 # mnist data input  (img shape: 28*28)     n_classes = 10 # mnist total classes  (0-9 digits)       # tf graph input   x = tf.placeholder ("float", [none,  N_input])    y = tf.placeholder ("float",  [none, n_classes])           # create model   Def multilayer_perceptron (x, weights,  biases):       # Hidden layer with RELU activation        layer_1 = tf.add (Tf.matmul (x, weights[' H1 '),  biases[' B1 '])    &nbsP;   layer_1 = tf.nn.relu (layer_1)        # hidden  layer with RELU activation       layer_2 = tf.add ( Tf.matmul (layer_1, weights[' H2 ']),  biases[' B2 ']        layer_2 =  tf.nn.relu (layer_2)        # output layer with linear  activation       out_layer = tf.matmul (layer_2, weights[' Out '])  + biases[' out ']       return out_layer       # store layers weight & bias   weights = {        ' H1 ':  TF. Variable (Tf.random_normal ([n_input, n_hidden_1])),        ' H2 ':  TF. Variable (Tf.random_normal ([n_hidden_1, n_hidden_2])),        ' Out ':  TF. Variable (Tf.random_normal ([n_hidden_2, n_classes]))   }   biases = {         ' B1 ':  TF. Variable (Tf.random_normal ([n_hidden_1])),        ' B2 ':  TF. Variable (Tf.random_normal ([n_hidden_2])),        ' out ':  TF. Variable (Tf.random_normal ([n_classes]))   }      # construct model    Pred = multilayer_perceptron (x, weights, biases)       #  define loss and optimizer   Cost = tf.reduce_mean (Tf.nn.softmax_cross_ Entropy_with_logits (pred, y))    Optimizer = tf.train.adamoptimizer (learning_rate= learning_rate). Minimize (cost)       # initializing the variables   Init = tf.initialize_all_variables ()       #  ' Saver '  op to save and restore all the variables   saver =  Tf.train.Saver ()       # running first session   print  "starting  1st session ... "   WITH TF. Session ()  as sess:       # Initialize variables       sess.run (init)           # Training  cycle       for epoch in range (3):            avg_cost = 0.            total_batch = int (mnist.train.num_examples/batch_size)             # Loop over all batches            for i in range (total_batch):    &NBsp;          batch_x, batch_y =  Mnist.train.next_batch (batch_size)                 # Run optimization op  (Backprop)  and cost op  (to get  loss value)                _,  c = sess.run ([optimizer, cost], feed_dict={x: batch_x,                                                                 y: batch_y})                 # compute average loss               avg_cost +=  c / total_batch           # display  logs per epoch step           if  epoch % display_step == 0:                print  "Epoch:",  '%04d '  %  (epoch+1),  "cost=", \                     "{:. 9f}" . Format (avg_cost)        print  "first optimization finished!"           # Test model        Correct_prediction = tf.equal (Tf.argmax (pred, 1),  tf.argmax (y, 1))         # calculate accuracy       accuracy = tf.reduce_mean (Tf.cast (correct_ prediction,  "float")        print  "accuracy:",  accuracy.eval ({x:  mnist.test.images, y: mnist.test.labels})           #  Save model weights to disk       save_path =  Saver.save (Sess, model_path)        print  "Model saved in  file: %s " % save_path      # running a new session    print  "Starting 2nd session ..."    WITH TF. Session ()  as sess:       # Initialize variables       sess.run (init)           # Restore  model weights from previously saved model       load_path = saver.restore (Sess, model _path)        print  "model restored from file: %s"  %  save_path          # Resume training        for epoch in range (7):            avg_cost = 0.           total_batch =  int (mnist.train.num_examples / batch_size)             # Loop over all batches            for i in range (total_batch):                batch_x, batch_y = mnist.train.next_batch (batch_size)                 # run optimization op  (Backprop)  and cost op  (to get loss value)                 _, c = sess.run ([Optimizer, cost], feed_ dict={x: batch_x,                                                                y: batch_y})                 # Compute average loss                avg_cost += c / total_ batch    &NBsp;      # display logs per epoch step           if epoch % display_step == 0:                print  "Epoch:",  '%04d '  %  (epoch + 1),  "cost=", \                     "{:. 9f}". Format (avg_cost)         print  "second optimization finished!"           # Test model        Correct_prediction = tf.equal (Tf.argmax (pred, 1),  tf.argmax (y, 1))         # Calculate accuracy       accuracy =  Tf.reduce_mean (Tf.cast (correct_prediction,  "float"))        print  "accuracy:",  accuracy.eval (            {x: mnist.test.images, y: mnist.test.labels})   

Three running results

Resources:

https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/4_Utils/save_restore_model.py
Https://www.tensorflow.org/versions/r0.9/api_docs/python/state_ops.html#Saver