Caffe Learning note 2:fine-tuning A class-aware preprocessing network

Fine-tuning a pre-processing network for type recognition (fine-tuning a pretrained network for Style recognition)

This is the original address here

In this experiment, we explored a common approach that was useful in real-world applications: Using an Caffe network that was trained in advance, and using custom data to fine-tune parameters.
The advantage of this method is that I trained well in advance of the fall is from a very large image data set to learn, the middle of the network can capture the general visual representation of the semantic information. Consider it as a very powerful general-purpose visualization feature that you can think of as a black box. Most importantly (on top of that), a relatively small amount of data is required to have a good performance on the target task.
First, we need to prepare the data. This includes the following steps:
1. Get the imagenet ILSVRC pre-trained model through the provided shell scripts.
2. Download a subset of the Flickr style datasets for this demo.
3. Compiling the downloaded Flickr data to Caffe is the data format.

Caffe_root ='. /'  # This file should is run from {caffe_root}/examples (otherwise)ImportSyssys.path.insert (0, Caffe_root +' python ')ImportCaffecaffe.set_device (0) Caffe.set_mode_gpu ()ImportNumPy asNp fromPylabImport*%matplotlib InlineImportTempfile# Helper function for deprocessing preprocessed images, e.g., for display. def deprocess_net_image(image):Image = Image.copy ()# don ' t modify destructivelyImage = image[::-1]# BGR RGBImage = Image.transpose (1,2,0)# CHW-HWCImage + = [123,117,104]# (approximately) undo mean subtraction    # clamp values in [0, 255]Image[image <0], Image[image >255] =0,255    # Round and cast from float32 to Uint8Image = Np.round (image) image = Np.require (image, Dtype=np.uint8)returnImage

1. Setup and data set download

Downloading data requires these executions

• get_ilsvrc_aux.sh to download imagenet data mean and label, etc.
• download_model_binary.py to download the pre-trained Reference Model
• finetune_flickr_style/assemble_data.py Download style training and test data

When we do this, we'll download a small fraction of all datasets: Just 2000 images from the 80k image, 5 from 20 style categories. (To get all datasets you can set full_dataset=true in the next settings)

# Download just a small subset ofThe data forThis exercise.# ( -  of  theK images,5  of  -Labels.) # toDownload the entire dataset, set ' Full_dataset = True '. Full_dataset = FalseifFull_dataset:num_style_images = Num_style_labels =-1Else: Num_style_images = -Num_style_labels =5# This downloads the ILSVRC auxiliary data (meanfile, etc), # andA subset of  -Images forThe style recognition Task.import Osos.chdir (caffe_root) # Run scripts from Caffe Root!data/ilsvrc12/get_ilsvrc_aux.sh!s cripts/download_model_binary.py Models/bvlc_reference_caffenet!python Examples/finetune_flickr_style/assemble_ data.py--workers=-1--seed=1701 \    --images= $NUM _style_images--label= $NUM _style_labels# back toExamplesos.chdir (' Examples‘)

Downloading ...
–2016-02-24 00:28:36–http://dl.caffe.berkeleyvision.org/caffe_ilsvrc12.tar.gz
Resolving dl.caffe.berkeleyvision.org (dl.caffe.berkeleyvision.org) ... 169.229.222.251
Connecting to Dl.caffe.berkeleyvision.org (dl.caffe.berkeleyvision.org) |169.229.222.251|:80 ... connected.
HTTP request sent, awaiting response ... OK
length:17858008 (17M) [Application/octet-stream]
Saving to: ' caffe_ilsvrc12.tar.gz '
100%[======================================>] 17,858,008 112mb/s in 0.2s
2016-02-24 00:28:36 (s)-' caffe_ilsvrc12.tar.gz ' saved [17858008/17858008]
Unzipping ...
Done.
Model already exists.
Downloading images with 7 workers ...
Writing Train/val for 1996 successfully downloaded images.

Definition weights , set the Imagenet pre-trained weight path we just downloaded to make sure it exists.

os‘models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel‘assertos.path.exists(weights)

From ilsvrc12/synset_words.txt loading 1000 imagenet tags, and finetune_flickr_style/style_names.txt loading 5 style tags from.

# Load ImageNet labels to imagenet_labelsImagenet_label_file = Caffe_root +' Data/ilsvrc12/synset_words.txt 'Imagenet_labels =List(Np.loadtxt (Imagenet_label_file, str, delimiter=' \ t ') Assert len (imagenet_labels) = = +Print ' Loaded ImageNet labels:\n ',' \ n '. Join (imagenet_labels[:Ten] + [' ... '])# Load Style labels to style_labelsStyle_label_file = Caffe_root +' Examples/finetune_flickr_style/style_names.txt 'Style_labels =List(Np.loadtxt (Style_label_file, str, delimiter=' \ n '))ifNum_style_labels >0: Style_labels = Style_labels[:num_style_labels]Print ' \nloaded style labels:\n ',', '. Join (Style_labels)

2. Define and run the network

We start by defining caffenet a function to initialize the caffenet structure (a small variant in alexnet), specifying the number of data and output classes with parameters.

 fromCaffeImportLayers asL fromCaffeImportParams asPweight_param = Dict (lr_mult=1, decay_mult=1) Bias_param = Dict (lr_mult=2, decay_mult=0) Learned_param = [Weight_param, Bias_param]frozen_param = [Dict (lr_mult=0)] *2 def conv_relu(bottom, KS, Nout, stride=1, pad=0, group=1,              Param=learned_param, Weight_filler=dict(type=' Gaussian ', std=0.01), Bias_filler=dict(type=' constant ', value=0.1)):CONV = l.convolution (bottom, Kernel_size=ks, Stride=stride, Num_output=nout, Pad=pad, Group=group, Param=param, Weight_filler=weight_filler, Bias_filler=bias_filler)returnConv, L.relu (CONV, in_place=True) def fc_relu(bottom, Nout, Param=learned_param, Weight_filler=dict(type=' Gauss Ian ', std=0.005), bias_filler=dict(type=' constant ', value=0.1) ):FC = L.innerproduct (bottom, Num_output=nout, Param=param, Weight_filler=weight_filler, Bias_filler=bias_filler)returnFC, L.relu (FC, in_place=True) def max_pool(bottom, KS, stride=1):    returnL.pooling (Bottom, Pool=p.pooling.max, Kernel_size=ks, Stride=stride) def caffenet(data, Label=none, Train=true, num_classes=, Classifier_nam e=' Fc8 ', Learn_all=false):    "" " Returns a Netspec specifying caffenet, following the original proto text specification (./models/bvlc_referen ce_caffenet/train_val.prototxt). "" "n = caffe.netspec () n.data = data param = Learned_paramifLearn_allElseFrozen_param n.conv1, n.relu1 = Conv_relu (N.data, One, the, stride=4, param=param) n.pool1 = Max_pool (N.RELU1,3, stride=2) N.norm1 = L.LRN (N.pool1, local_size=5, alpha=1e-4, Beta=0.75) n.conv2, N.RELU2 = Conv_relu (N.norm1,5, the, pad=2, group=2, param=param) n.pool2 = Max_pool (N.RELU2,3, stride=2) N.norm2 = L.LRN (N.pool2, local_size=5, alpha=1e-4, Beta=0.75) n.conv3, N.RELU3 = Conv_relu (N.norm2,3,384, pad=1, Param=param) n.conv4, N.relu4 = Conv_relu (N.RELU3,3,384, pad=1, group=2, Param=param) n.conv5, n.relu5 = Conv_relu (N.relu4,3, the, pad=1, group=2, param=param) N.pool5 = Max_pool (N.relu5,3, stride=2) n.fc6, N.relu6 = Fc_relu (N.POOL5,4096, Param=param)ifTRAIN:N.DROP6 = Fc7input = L.dropout (N.relu6, in_place=True)Else: Fc7input = N.relu6 n.fc7, N.relu7 = Fc_relu (Fc7input,4096, Param=param)ifTRAIN:N.DROP7 = Fc8input = L.dropout (N.relu7, in_place=True)Else: Fc8input = N.relu7# Always learn Fc8 (Param=learned_param)Fc8 = L.innerproduct (Fc8input, num_output=num_classes, Param=learned_param)# give Fc8 the name specified by argument ' Classifier_name 'N.__setattr__ (Classifier_name, FC8)if  notTrain:n.probs = L.softmax (FC8)ifLabel is  not None: N.label = Label N.loss = L.softmaxwithloss (Fc8, n.label) N.ACC = L.accuracy (Fc8, N.label)# Write the net to a temporary file and return its filename     withTempfile. Namedtemporaryfile (delete=False) asF:f.write (str (N.to_proto ()))returnF.name

Now, let's create a caffenet network that takes the "dummy data" without the tag as input, allowing us to set the input image from the outside and see what sort of imagenet is predicted.

dummy_data = L.DummyData(shape=dict(dim=[1, 3, 227, 227]))imagenet_net_filename = caffenet(data=dummy_data, train=False)imagenet_net = caffe.Net(imagenet_net_filename, weights, caffe.TEST)

Defines a function style_net that is called caffenet .

This new network will also have a caffenet structure, with different inputs and outputs.

• The input is the Flickr style we downloaded and was entered by a imagedata layer.
• The output is a 20 class distribution instead of the original 1000 imagenet classes
• The classification layer is renamed fc8_flickr instead fc8 , telling Caffe not to load the original classification (FC8) weights from the pre-trained imagenet model.

 def style_net(train=true, Learn_all=false, Subset=none):    ifSubset is None: subset =' Train ' ifTrainElse ' Test 'Source = Caffe_root +' Data/flickr_style/%s.txt '% subset Transform_param = Dict (Mirror=train, crop_size=227, Mean_file=caffe_root +' Data/ilsvrc12/imagenet_mean.binaryproto ') Style_data, Style_label = L.imagedata (Transform_param=transform_param, Source=source, batch_size= -, new_height= the, new_width= the, ntop=2)returnCaffenet (Data=style_data, Label=style_label, Train=train, Num_classes=num_style_labels, Classifier_name=' Fc8_flickr ', Learn_all=learn_all)

Caffe Learning note 2:fine-tuning A class-aware preprocessing network