Tensorflow-slim Learning Notes (ii) the first level catalogue code reading _ machine learning

Http://www.cnblogs.com/bmsl/p/dongbin_bmsl_02.html

By reading code to learn, always the most direct and fast. This chapter will explain the code for the first level of slim directory Tensorflow/tensorflow/contrib/slim/python/slim.

This layer of code mainly includes learning.py, evaluation.py, summary.py, queue.py and model_analyzer.py, respectively corresponding to the model training, testing, logging, queue Management and Model Analysis section.

Let Bameso tears to take you from easy to difficult to read the code. The following items are basically sorted by difficulty.

1. Model Analysis Module model_analyzer.py

This code provides the tools to analyze the operations and variables in the TensorFlow diagram.

#to analyze the operations in a graph:

Images, labels = loaddata (...)
Predictions = MyModel (IMAGES)

Slim.model_analyzer.analyze_ops (Tf.get_default_graph (), print_info=true)

#to analyze the model variables in a graph:

variables = Tf.model_variables ()

Slim.model_analyzer.analyze_vars (variables, print_info=false)

2. queue.py provides the queue content management code

3. summaries.py includes an auxiliary function to generate the day to record, summaries_test.py is one of its tests, using the example below:

Import TensorFlow as TF
Slim = Tf.contrib.slim

Slim.summaries.add_histogram_summaries (Slim.variables.get_model_variables ())
Slim.summaries.add_scalar_summary (Total_loss, ' total loss ')
Slim.summaries.add_scalar_summary (learning_rate, ' learning rate ')
Slim.summaries.add_histogram_summaries (my_tensors)
Slim.summaries.add_zero_fraction_summaries (my_tensors)

4. evaluation.py generally do not use slim code, so ignore.

5. The most important training model of the code in learning.py, where learning_test.py is a test example

The learning.py includes several functions required by the training model, including the operating gradient, generating a training operation (TRAIN_OP, an operation to compute the loss and applying gradients), and a training loop function. The training cycle requires the user to pass in the train_op to optimize. Note that the training loop here uses the Tf.train.Supervisor and the managed_session in its code implementation, which ensures that the machine can resume training from a failed run. Supervisor is an encapsulation of the coordinator,saver and SessionManager responsible for training programs. The following is a simple example:

# Load data and create the model:

Images, labels = loaddata (...)
Predictions = MyModel (IMAGES)

# Define The loss:
Slim.losses.log_loss (predictions, labels)
Total_loss = Slim.losses.get_total_loss ()

# Define the optimizer:
Optimizer = Tf.train.MomentumOptimizer (Flags.learning_rate, Flags.momentum)

# Create The Train_op
Train_op = Slim.learning.create_train_op (Total_loss, optimizer)

# Run Training.
Slim.learning.train (Train_op, My_log_dir)

The training must first define a TRAIN_OP, its function includes: (1) calculates the loss (2) applies the gradient to the parameter the update (3) returns the loss value

Train_op sometimes have to perform additional non gradient updates, such as batch_norm, during training. Batch_norm needs to perform a series of non-gradient update operations during training. Slim.learning.create_train_op allows the user to pass both the Update_ops list and the gradient update operation to the function at the same time.

Train_op = Slim.learning.create_train_op (Total_loss, Optimizer, Update_ops)

By default, Slim.learning.create_train_op includes all in TF. Graphkeys.update_ops the update operation in the collection. Also, the slim slim.batch_norm function adds moving mean and moving variance updates to this collection. Therefore, users who use Slim.batch_norm can ensure that updates to moving mean and moving variance are correctly computed without considering the extra steps of the task. If the user needs to add additional updates, you can use the following code:

# Force Tf-slim not ' to ' any update_ops:
Train_op = Slim.learning.create_train_op (
Total_loss,
Optimizer
Update_ops=[])

# Use an alternative set of update Ops:
Train_op = Slim.learning.create_train_op (
Total_loss,
Optimizer
Update_ops=my_other_update_ops)

# Use a alternative set of update OPS in addition to the default updates:
Tf.add_to_collection (TF. Graphkeys.update_ops, MY_UPDATE0)
Tf.add_to_collection (TF. Graphkeys.update_ops, My_update1)

Train_op = Slim.learning.create_train_op (
Total_loss,
Optimizer

# Which is the same as:
Train_op = Slim.learning.create_train_op (
Total_loss,
Optimizer
Update_ops=tf.get_collection (TF. Graphkeys.update_ops))

The most important of learning training is two functions: Create_train_op and Train

(1) def create_train_op (Total_loss,
Optimizer
Global_step=_use_global_step,
Update_ops=none,
Variables_to_train=none,
Clip_gradient_norm=0,
Summarize_gradients=false,
Gate_gradients=tf_optimizer. Optimizer.gate_op,
Aggregation_method=none,
Colocate_gradients_with_ops=false,
Gradient_multipliers=none,
Check_numerics=true)

Clip_gradient_norm: When it is greater than 0 o'clock, the gradient is due to this value.

The return value is a tensor that calculates the gradient and returns the loss value.

(2) def train (Train_op,

LogDir,
Train_step_fn=train_step,
Train_step_kwargs=_use_default,
Log_every_n_steps=1,
Graph=none,
Master= ',
Is_chief=true,
Global_step=none,
Number_of_steps=none,
Init_op=_use_default,
Init_feed_dict=none,
Local_init_op=_use_default,
Init_fn=none,
Ready_op=_use_default,
Summary_op=_use_default,
save_summaries_secs=600,
Summary_writer=_use_default,
Startup_delay_steps=0,
Saver=none,
save_interval_secs=600,
Sync_optimizer=none,
Session_config=none,
Trace_every_n_steps=none)

This function runs a training loop using TensorFlow's supervisor. When using Sync_optimizer, the gradient calculation is performed synchronously. Otherwise, the gradient computation is performed asynchronously.

Parameters:

Log_every_n_steps: Loss function and step's printing frequency

Master:tensorflow Master's address

Is_chief: Specifies whether the training is implemented with a master replica when multiple copies are being trained.

Sync_optimizer: a TF. Syncreplicasoptimizer or a list of them. If this parameter is provided, the gradient update is synchronized, and if none is used, the update is asynchronous.