Caffe Combat series: to achieve their own Caffe network layer

Due to the introduction of a previous article on the implementation of their own network layer, but the article difficult, this time I have the simplest image scaling layer for example to implement.

Before you explain, there are a few prerequisites you need to master, and that is that you already know how to install Caffe and the directories inside Caffe.

First of all, we design the parameters of our layer out_height, that is, the output of the image of the height out_width, that is, the output image width visualize, whether the image needs to be displayed

Then you can add the following code to the Src/caffe/proto/caffe.proto file:

Message Imagescaleparameter {
  //Specify the output height and width
  optional UInt32 out_height = 1;
  Optional UInt32 out_width = 2;
 
  For debug you can check the source images and scaled images
  optional BOOL visualize = 3 [default = False];
}

This specifies the name of the parameter and the type of the parameter, optional that the parameter is optional and may not appear, and [Default=false] indicates that the default value of the parameter is false. Each parameter specifies a number indicating the identity of the parameter.
Next, we can put our design parameters into the Layerparameter:

Optional Hingelossparameter Hinge_loss_param = 114;
Optional Imagedataparameter Image_data_param =;
Optional Imagescaleparameter Image_scale_param = 147;
Optional Infogainlossparameter Infogain_loss_param = 116;
Optional Innerproductparameter Inner_product_param = 117;

Note that when you join, take a look at Layerparameter's comments, and when you are done, you should also pay attention to adding such hints, so that it is easier for future generations to add custom layer//Layerparameter next available layer-specific ID: 148 (last Added:image_scale_param)
Next we implement the header file for our own layer:
(1) The implementation of the first need to set not allow the header file to repeat the inclusion of macro definitions:

#ifndef caffe_image_scale_layer_hpp_
#define Caffe_image_scale_layer_hpp_

(2) Add the necessary header files

#include "caffe/blob.hpp"
#include "caffe/layer.hpp"
#include "caffe/proto/caffe.pb.h"
#include "Caffe /LAYER.HPP "

(3) The type string that joins the returned layer

Virtual Inline const char* type () const {return "Imagescale";}

(4) Tell Caffe the input of this layer has several, output has several

Virtual inline int exactnumbottomblobs () const {return 1;}
Virtual inline int exactnumtopblobs () const {return 1;}

(5) Because the implementation of this layer is the image scaling, so do not need to reverse transmission, so directly write an empty virtual function of the implementation

virtual void backward_cpu (const vector<blob<dtype>*>& Top,
  const vector<bool>& Propagate_down, const vector<blob<dtype>*>& bottom) {};

(6) Define the member variables in the class used during use, note that the name of the class's member variable is the end of the underscore, which keeps the code consistent with the Caffe

  int out_height_;
  int out_width_;
  int height_;
  int width_;
  BOOL Visualize_;
  int Num_images_;
  int Num_channels_;

(7) Finally, don't forget to join the ENDIF this macro, and also pay attention to adding the necessary comments to indicate what this endif corresponds to

#endif  //Caffe_image_scale_layer_hpp_

The following is a detailed header file code:

#ifndef caffe_image_scale_layer_hpp_ #define CAFFE_IMAGE_SCALE_LAYER_HPP_ #include "caffe/blob.hpp" #include "caffe/ 
Layer.hpp "#include" caffe/proto/caffe.pb.h "#include" caffe/layer.hpp "namespace Caffe {//written by xizero00 2016/9/13 Template <typename dtype> class imagescalelayer:public layer<dtype> {public:explicit ImageScaleLayer (c Onst layerparameter& param): layer<dtype> (param) {} virtual void Layersetup (const vector<blob<dty
  pe>*>& Bottom, const vector<blob<dtype>*>& top);  virtual void reshape (const vector<blob<dtype>*>& bottom, const vector<blob<dtype>*>&
  top);
  Virtual Inline const char* type () const {return "Imagescale";}
  Virtual inline int exactnumbottomblobs () const {return 1;}
 Virtual inline int exactnumtopblobs () const {return 1;} Protected:///@copydoc imagescalelayer virtual void forward_cpu (const vector<blob<dtype>*>& boTtom, const vector<blob<dtype>*>& top); virtual void backward_cpu (const vector<blob<dtype>*>& Top, const vector<bool>& propagate_d
  Own, const vector<blob<dtype>*>& bottom) {};
  int out_height_;
  int out_width_;
  int height_;
  int width_;
  BOOL Visualize_;
  int Num_images_;
int Num_channels_;
}; }//Namespace Caffe #endif//Caffe_image_scale_layer_hpp_

Next, write the specific layer settings and the implementation of the layer's forward Transmission: (8) Add the necessary header files

#include "caffe/layers/image_scale_layer.hpp"
#include "caffe/util/math_functions.hpp"
#include < Opencv2/opencv.hpp>

(9) Implementation of the level of the Set function Layersetup, in the function of the network configuration parameters read into the class member variables, easy to forward the time and the layer to set the time to use, and check the legality of the parameters

Template <typename dtype>
void Imagescalelayer<dtype>::layersetup (const vector<blob<dtype >*>& Bottom,
      const vector<blob<dtype>*>& top) {
  //Get Parameters
  const imagescaleparameter& param = This->layer_param_.image_scale_param ();
  Get the output size
  Out_height_ = Param.out_height ();
  Out_width_ = Param.out_width ();
  Visualize_ = Param.visualize ();
 
  Get the input size
  Num_images_ = Bottom[0]->num ();
  Height_ = Bottom[0]->height ();
  Width_ = Bottom[0]->width ();
  Num_channels_ = Bottom[0]->channels ();
  Check the channels must is images
  //channel must be 1 or 3, gray image or color image
  Check_eq (num_channel s_==3) | | (Num_channels_ = = 1), true);
  Check the output size
  check_gt (out_height_, 0);
  CHECK_GT (out_height_, 0);
 
}

(10) To implement the reshape function of the layer to set the output size of the layer, we use the parameter class from the network configuration file to set the output size

Template <typename dtype>
void Imagescalelayer<dtype>::reshape (const vector<blob<dtype>* >& Bottom,
      const vector<blob<dtype>*>& top) {
  //reshape the outputs
  top[0]-> Reshape (Num_images_, Num_channels_, Out_height_, out_width_);

(11) To achieve the forward propagation function forward_cpu, I realized that the image is scaled one by one to the size given in the configuration file.

Template <typename dtype> void imagescalelayer<dtype>::forward_cpu (const vector<blob<dtype>*& 
  gt;& Bottom, const vector<blob<dtype>*>& top) {Const dtype* bottom_data = Bottom[0]->cpu_data ();
  Dtype * Top_data = Top[0]->mutable_cpu_data ();
 
  Cv::mat Srcimage, Dstimage;
  Precompurte the index const int srcimagesize = Width_ * HEIGHT_;
  const int dstimagesize = Out_width_ * OUT_HEIGHT_;
  const int srcchimagesize = srcimagesize * NUM_CHANNELS_;
  const int dstchimagesize = dstimagesize * NUM_CHANNELS_;
        for (int idx_img = 0; idx_img < Num_images_ idx_img++) {//Zeros source images and scaled images
        Srcimage = Cv::mat::zeros (Height_, Width_, CV_32FC1);
        Dstimage = Cv::mat::zeros (Out_height_, Out_width_, CV_32FC1);  Read from bottom[0] for (int idx_ch = 0; idx_ch < Num_channels_; idx_ch++) {for
                (int i = 0; i < height_; i++) {
                        for (int j=0; J < Width_; J + +) {int
                                IMAGE_IDX = idx_img * srcchimagesize + srcimagesize * idx_ch + height_ + j;
                        Srcimage.at<float> (i,j) = (float) bottom_data[image_idx]; }}//resize to specified size//We-use linear interpolation CV::
        Resize (srcimage, Dstimage, Dstimage.size ());
                Store the resized image to top[0] for (int idx_ch = 0; idx_ch < Num_channels_; idx_ch++) { for (int i = 0; i < Out_height_. i++) {for (int j = 0; J < out_width _; J + +) {int image_idx = idx_img * dstchimagesize + dstimagesize * I
                                Dx_ch + Out_height_*i + j;
                       TOP_DATA[IMAGE_IDX] = dstimage.at<float> (I,J); }} if (Visualize_) {Cv::namedwindow ("src image", Cv_window_au
                Tosize);
                Cv::namedwindow ("DST image", cv_window_autosize);
                Cv::imshow ("src image", srcimage);
                Cv::imshow ("DST image", dstimage);
        Cv::waitkey (0); }
  }
}

Finally, a complete implementation is given:

#include "caffe/layers/image_scale_layer.hpp" #include "caffe/util/math_functions.hpp" #include <opencv2/ Opencv.hpp> namespace Caffe {Template <typename dtype> void Imagescalelayer<dtype>::layersetup (const 
  vector<blob<dtype>*>& Bottom, const vector<blob<dtype>*>& top) {//Get parameters
  Const imagescaleparameter& param = This->layer_param_.image_scale_param ();
  Get the output size Out_height_ = Param.out_height ();
  Out_width_ = Param.out_width ();
 
  Visualize_ = Param.visualize ();
  Get the input size Num_images_ = Bottom[0]->num ();
  Height_ = Bottom[0]->height ();
  Width_ = Bottom[0]->width ();
  Num_channels_ = Bottom[0]->channels ();  Check the channels must is images//channel must be 1 or 3, gray image or color image check_eq ((num_channels_==3) ||
  (Num_channels_ = = 1), true);
  Check the output size check_gt (out_height_, 0);
 
CHECK_GT (out_height_, 0); } template <typenAme dtype> void Imagescalelayer<dtype>::reshape (const vector<blob<dtype>*>& Bottom, const v ector<blob<dtype>*>& top) {//reshape the outputs Top[0]->reshape (Num_images_, Num_channels_, Out_
Height_, Out_width_); } template <typename dtype> void imagescalelayer<dtype>::forward_cpu (const vector<blob<dtype>*& 
  gt;& Bottom, const vector<blob<dtype>*>& top) {Const dtype* bottom_data = Bottom[0]->cpu_data ();
  Dtype * Top_data = Top[0]->mutable_cpu_data ();
 
  Cv::mat Srcimage, Dstimage;
  Precompurte the index const int srcimagesize = Width_ * HEIGHT_;
  const int dstimagesize = Out_width_ * OUT_HEIGHT_;
  const int srcchimagesize = srcimagesize * NUM_CHANNELS_;
  const int dstchimagesize = dstimagesize * NUM_CHANNELS_;
        for (int idx_img = 0; idx_img < Num_images_ idx_img++) {//Zeros source images and scaled images Srcimage = Cv::mat::zeros (Height_, WIDth_, CV_32FC1);
        Dstimage = Cv::mat::zeros (Out_height_, Out_width_, CV_32FC1);  Read from bottom[0] for (int idx_ch = 0; idx_ch < Num_channels_; idx_ch++) {for
                (int i = 0; i < height_; i++) {for (int j=0 J < Width_; J + +) {in
                                T image_idx = idx_img * srcchimagesize + srcimagesize * idx_ch + height_ + j;
                        Srcimage.at<float> (i,j) = (float) bottom_data[image_idx]; }}//resize to specified size//We-use linear interpolation CV::
        Resize (srcimage, Dstimage, Dstimage.size ());
                Store the resized image to top[0] for (int idx_ch = 0; idx_ch < Num_channels_; idx_ch++) { for (int i = 0; i < Out_height_. i++) {for (int j = 0; J < oUt_width_; J + +) {int image_idx = idx_img * dstchimagesize + dstimagesize * I
                                Dx_ch + Out_height_*i + j;
                        TOP_DATA[IMAGE_IDX] = dstimage.at<float> (I,J); }} if (Visualize_) {Cv::namedwindow ("src image", Cv_window_aut
                Osize);
                Cv::namedwindow ("DST image", cv_window_autosize);
                Cv::imshow ("src image", srcimage);
                Cv::imshow ("DST image", dstimage);
        Cv::waitkey (0);
}} #ifdef cpu_only Stub_gpu (Imagescalelayer);
#endif Instantiate_class (Imagescalelayer);
Register_layer_class (Imagescale);
 }//Namespace Caffe

Please save the above code as IMAGE_SCALE_LAYER.HPP and CPP. Then put them in the corresponding include and Src/caffe/layers folders.
Then you can configure the following when you use it

Layer {
  name: "imagescaled"
  type: "Imagescale"
  Bottom: "Data" top
  : "imagescaled"
  Image_scale_ param {
    out_height:128
    out_width:128
    visualize:true
  }
}

The Out_height and out_width in the above configuration are the size of the scaled picture, and the visualize indicates whether it is displayed or not. At this point, we have completed a very simple Caffe custom layer implementation, how, very simple. The model I tested (I think you definitely know how to convert a Mnist dataset to Lmdb by using the Caffe tool) is:

# Simple Single-layer network to showcase editing model parameters.
Name: "Sample"
Layer {
  name: "Data" type: "Data" top
  : "Data"
  include {
    phase:train
  }
  Transform_param {
    scale:0.0039215684
  }
  data_param {
    Source: "Examples/mnist/mnist_train_ Lmdb '
    batch_size:10
    backend:lmdb
  }
}

layer {
  name: ' imagescaled '
  type: ' Imagescale "
  bottom:" Data "top
  :" imagescaled "
  image_scale_param {
    out_height:128
    out_ width:128
    visualize:true
  }
}

The solver.prototxt used in the test

NET: "Examples/imagescale/sample.prototxt"


base_lr:0.01
lr_policy: "Step"
gamma:0.1
stepsize : 10000
display:1
max_iter:1
weight_decay:0.0005 snapshot:1 snapshot_prefix
: "examples/ Imagescale/sample "
momentum:0.9
# solver mode:cpu or GPU
Solver_mode:gpu

And then run it just to write a bash file to the Caffe directory:

#!/usr/bin/env sh
set-e

snap_dir= "examples/imagescale/snapshots"

mkdir-p $snap _dir

tools=./build /tools


$TOOLS/caffe train \
--solver=examples/imagescale/solver.prototxt 2>&1 | tee-a $snap _dir/ Train.log

Here are my results:

The smaller is the original image that was entered, the larger one is the scaled image.
Well, that's it.
Code package download, please stamp here http://download.csdn.net/detail/xizero00/9629898