Keras Introductory Lesson 5--Network visualization and training monitoring

Keras Introductory Lesson 5: Network Visualization and training monitoring

This section focuses on the visualization of neural networks in Keras, including the visualization of network structures and how to use Tensorboard to monitor the training process.
Here we borrow the code from lesson 2nd for examples and explanations.

The definition of the front of the network, data initialization is the same, mainly the Fit function is enabled Tensorboard

By adding the Tensorboard callback function to the FIT function of the model, the training data is automatically saved in the Log_dir specified directory, and then the command line is started Tensorboard–logdir=./log. Tensorboard will record the values in loss and model.metrics, in this case Acc,loss,val_acc,val_loss four values, updated each epoch.
In addition to these scalars, will also record the network graph, directly visualize the network structure, but compared with the native TensorFlow generated diagram, the difference is relatively large, rather ugly, so it is not recommended to use Tensorboard to view the network structure in Keras.

I only trained 2 epochs, so I recorded only two values. The graph is as follows

↓ histogram, used to statistic the distribution of parameters

Import Keras from
keras.datasets import mnist from
keras.models import sequential from
keras.layers import Dense, dropout, Flatten from
keras.layers import conv2d, maxpooling2d from
keras import backend as K
# introduced Tenso Rboard from
keras.callbacks import tensorboard from
keras.utils import Plot_model

(X_train,y_train), (x_ test,y_test) = Mnist.load_data () # out:np.ndarray

x_train = X_train.reshape ( -1,28,28,1)
x_test = X_ Test.reshape ( -1,28,28,1)
Input_shape = (28,28,1)

x_train = x_train/255 x_test
= x_test/255
y_ Train = keras.utils.to_categorical (y_train,10)
y_test = keras.utils.to_categorical (y_test,10)

Using TensorFlow backend.
/usr/local/cellar/python3/3.6.2/frameworks/python.framework/versions/3.6/lib/python3.6/importlib/_bootstrap.py : 205:runtimewarning:compiletime version 3.5 of module ' Tensorflow.python.framework.fast_tensor_util ' does not match run Time Version 3.6
  return F (*args, **kwds)

Model = sequential ()
Model.add (conv2d (filters = 32,kernel_size= (3,3),
                 activation= ' Relu ', Input_shape = Input_ Shape,name= ' Conv1 '))
Model.add (conv2d ((3,3), activation= ' Relu ', name= ' conv2 '))
Model.add (maxpooling2d (Pool_size= (2,2), name= ' pool2 '))
Model.add (Dropout (0.25,name= ' dropout1 '))
Model.add (Flatten (name= ' flat1 '))
model.add (dense (128, activation= ' Relu '))
Model.add (Dropout (0.5,name= ' dropout2 '))
model.add (Dense (10,activation= ' Softmax '), Name= ' output '))

Plot_model (model,to_file= ' model.png ')

↑keras's utils has a plot_model function that is used to visualize the structure of the network, in order to ensure that the format is beautiful, we define the model at the time of each layer has added a name.
For most keras layers, there is the name parameter.
Using Plot_model, you can generate a picture similar to the one below, which is much clearer than the Tensorboard graph. Therefore, it is recommended to use Keras to print the diagram structure in Keras.

Model.compile (loss = keras.losses.categorical_crossentropy,
             optimizer = Keras.optimizers.Adadelta (),
             metrics=[' accuracy ')

Tensorboard interface functions, there are many parameters to choose from, the details can be found in the official documents. Tensorboard use in Keras is simpler than summary save in TensorFlow, but is less flexible and is only suitable for some of the most basic uses.

TB = Tensorboard (log_dir= './logs ',  # log directory
                 histogram_freq=1,  # by what frequency (epoch) to calculate the histogram, 0 is not calculated
                 Batch_ size=32,     # Calculate the histogram write_graph=true with a lot of data,  # whether to store the network structure diagram
                 Write_grads=false, # Whether to visualize the gradient histogram
                 Write_ images=false,# whether visual parameters
                 embeddings_freq=0, 
                 embeddings_layer_names=none, 
                 embeddings_metadata=none)
callbacks = [TB]

Model.fit (x_train,y_train,batch_size=64,epochs=2
          , Verbose=1,validation_data= (x_test,y_test),
          Callbacks=callbacks)

SummaryLearn how to use the Tensorboard monitoring training process to learn how to save a network diagram using Keras's own Save_model function

When using Plot_model to draw a picture of the time also encountered some errors, if you also report an error, you can refer to my other article to try to solve: Mac use Keras Plot_model function Error Solution

This code address: HTTPS://GITHUB.COM/TSYCNH/KERAS-TUTORIALS/BLOB/MASTER/CLASS_5.IPYNB

Reference:

https://keras.io/visualization/