train the model and save the log file
Start by building a script file for the training data train.sh, which reads as follows, where 2>&1 | Tee Examples/mnist/mnist_train_log.log is the log log file's Save directory.
#!/usr/bin/env sh
set-e
tools=./build/tools
$TOOLS/caffe train --solver=examples/mnist/lenet_ Solver.prototxt 2>&1 | tee examples/mnist/mnist_train_log.log
After training is completed, the Mnist_train_log.log log is generated under the Examples/mnist folder
Drawing Curves
First, the folder Caffe/tools/extra under the parse_log.sh, extract_seconds.py, Plot_training_log.py.example copied to the previous log log file in the Save directory. Then, change the plot_training_log.py.example to plot_training_log.py and execute the following command to draw the curve:
Python plot_training_log.py 6 train_loss.png Mnist_train_log.log
Caffe supports a variety of curve plots, specifying different type parameters, with the following specific parameters:
Notes:
1. Supporting multiple logs.
2. log file name must end with the lower-cased ". Log".
Supported chart types:
0:test accuracy vs. Iters
1:test accuracy vs. Seconds
2:test loss vs. I ters
3:test loss vs. Seconds
4:train Learning rate vs. Iters 5:train
Learning rate vs. Se Conds
6:train loss vs. Iters
7:train Loss
Problems encountered:
Question 1: At the beginning when the type parameter was found to be 0-3, the curve can be drawn normally, but when the type parameter is 4-7, the following error occurs:
Workaround: After opening the generated Mnist_train_log.log.train file, it is found that there is a problem with extracting the Trainingloss data, as shown in Figure 1; Later, click on the blog to open the link in the method to modify the parse_log.sh. For parse-log.sh generated files Mnist_train_log.log.train,caffe the parse-log.sh extracted to the 9th column (9th field), which is the number with S/10, I changed it to the 13th field, That is, the number after the loss and = symbol, the real loss in the 13th column, as shown in Figure 2, 3.
Figure 1
Figure 2
Figure 3
Issue 2: After solving the above problem, the type parameter is 4-7, still can not draw the curve, the following problem occurs: Indexerror:list index out of range
Solution: The big God in the group to help find the problem, it is the last row of M nist_train_log.log.train data missing problem, helpless, log file really no data, can only forge a set of data, as shown below, the red circle is my own Add. However, do not feel that this is not good, about this problem, we have encountered no, how to solve, you can exchange a bit.
Pre-Modified:
Modified by: