Training and testing your own pictures under Linux __linux

The goal of learning Caffe is not simply to do a few exercises, but to use your own actual projects or scientific research in the end. Therefore, this article describes the entire process from your own original picture to the Lmdb data to the training and testing model.

I. Preparation of data

Conditional students, you can go to Imagenet's official website http://www.image-net.org/download-images, download imagenet pictures to train. But I did not download, a reason is registered account, the verification code always come out (heard is the Google site verification code, and I was not on Google). The second reason is that the data is too big ...

I went online to find some other pictures to replace, a total of 500 pictures, divided into buses, dinosaurs, elephants, flowers and horses five classes, each class 100. Need the classmate, can go to My network disk download: Http://pan.baidu.com/s/1nuqlTnN

The numbering begins with 3,4,5,6,7, each of which is a class. I selected 20 pieces from each of them as a test and the remaining 80 as training. Therefore the final training picture 400, the test picture 100, altogether 5 class. I placed the picture under the Caffe root directory under the Data folder. The Training Picture catalog: data/re/train/, test picture catalog: data/re/test/

Second, convert to Lmdb format

Specific conversion process, see my previous post: Caffe Learning Series (11): Image data conversion to db (Leveldb/lmdb) file

First, create a myfile folder under examples to store the configuration and script files. Then write a script create_filelist.sh to generate the Train.txt and test.txt manifest files

# sudo mkdir examples/myfile
# sudo vi examples/myfile/create_filelist.sh

Edit the file, write the following code, and save the

#!/usr/bin/env sh
data=data/re/
my=examples/myfile

echo "Create train.txt ..."
rm-rf $MY/train.txt for
i in 3 4 5 6 7 does find
$DATA/train-name $i *.jpg | Cut-d '/'-f4-5 | Sed "s/$/$i/" >> $MY/train.txt
done
echo "Create test.txt ..."
rm-rf $MY/test.txt for
i in 3 4 5 6 7 Do find
$DATA/test-name $i *.jpg | cut-d '/'-f4-5 | sed s/$/$i/">> $MY/test.txt doing
echo" A ll Done "

Then, run this script

# sudo sh examples/myfile/create_filelist.sh

Successful, you will be in the examples/myfile/folder to generate Train.txt and test.txt two text files, which is the list of pictures.

Then write a script file that invokes the Convert_imageset command to transform the data format.

# sudo VI examples/myfile/create_lmdb.sh

Insert:

#!/usr/bin/env sh
my=examples/myfile

echo "Create train Lmdb ..."
RM-RF $MY/img_train_lmdb
build/tools/convert_imageset \
--shuffle \
--resize_height=256 \
--resize _width=256 \
/home/xxx/caffe/data/re/\
$MY/train.txt \
$MY/img_train_lmdb

echo "Create test Lmdb ..."
rm-rf $MY/img_test_lmdb
build/tools/convert_imageset \
--shuffle \
--resize_width=256 \
--resize_height=256 \
/home/xxx/caffe/data/re/\
$MY/test.txt \
$MY/img_test_lmdb

echo "all Done ... "

Because the picture size is different, therefore I unify transforms into the 256*256 size. After successful operation, two folders Img_train_lmdb and Img_test_lmdb are generated under Examples/myfile to save the Lmdb file after the picture transformation.

Three, calculate the mean value and save

Images minus mean retraining will improve the speed and accuracy of training. Therefore, this operation is generally available.

Caffe Program provides a means of calculating the mean value of the file Compute_image_mean.cpp, we can use it directly

# sudo build/tools/compute_image_mean examples/myfile/img_train_lmdb examples/myfile/mean.binaryproto

Compute_image_mean with two parameters, the first parameter is the LMDB training data location, the second parameter sets the name of the mean file and the save path.
After successful operation, a mean.binaryproto mean file is generated under examples/myfile/.

Create a model and write a configuration file

The model uses the Caffenet model with the program, which is located in the models/bvlc_reference_caffenet/folder, and copies the required two profiles to the MyFile folder

# sudo cp models/bvlc_reference_caffenet/solver.prototxt examples/myfile/
# sudo cp models/bvlc_reference_ Caffenet/train_val.prototxt examples/myfile/

Modify the Solver.prototxt

# sudo VI examples/myfile/solver.prototxt

NET: "Examples/myfile/train_val.prototxt"
test_iter:2
test_interval:50
base_lr:0.001
lr_policy: "Step"
gamma:0.1
stepsize:100
display:20
max_iter:500
momentum:0.9
weight_decay:0.005
Solver_mode:gpu

100 test data, Batch_size 50, so Test_iter set to 2, you can cover the whole. In the course of training, adjust the learning rate, gradually become smaller.

Modify the Train_val.protxt, only need to modify the two stages of the data layer on it, others can be used without control.

Name: ' caffenet '
layer {
  name: ' Data '
  type: ' Data ' top
  : ' Data ' top
  : ' label '
  include {
    Phase:train
  }
  transform_param {
    mirror:true
    crop_size:227
    mean_file: "examples/myfile/ Mean.binaryproto '
  }
  data_param {
    Source: ' Examples/myfile/img_train_lmdb '
    batch_size:256
    Backend:lmdb
  }
}
Layer {
  name: ' Data ' type: ' Data ' top
  : ' Data ' top
  : ' label '
  include {
    phase:test
  }
  Transform_param {
    mirror:false
    crop_size:227
    mean_file: "Examples/myfile/mean.binaryproto"
  }
  Data_param {
    Source: "Examples/myfile/img_test_lmdb"
    batch_size:50
    backend:lmdb
  }
}

In fact, it is to modify the two data layer of the mean_file and source of these two places, the other has not changed.

V. Training and Testing

If there is no problem before, the data is ready, the configuration file is also configured, this step is relatively simple.

# sudo Build/tools/caffe train-solver examples/myfile/solver.prototxt

The running time and the final precision will vary according to machine configuration and parameter settings. Mine is GPU+CUDNN run 500 times, about 8 minutes, with a precision of 95%.