Caffe Source Reading 4-layer.hpp

An interface for the

units of computation which can is composed into a Net.

Layer&s must implement a Forward function, in which they take their input (bottom) blob&s (if any) and compute the IR Output blob&s (if any). They also implement a backward function, in which they compute the error gradients with respect to their input blob&am P;s, given the error gradients with their output blob&s.

Follow our general convolution neural network model to understand: a network (net) contains a lot of layers (layer), and the contents of each layer are not outside the data (feedforward data and feedback error), the data in the Caffe has been implemented with the Blob class, it should be possible to think that the layer should contain a lot of blob class, Or a vector of a blob type.

What are the data in the 1 layer?

Protected:/** The PROTOBUF that stores the layer parameters */  layerparameter LAYER_PARAM_;  /** the vector that stores the learnable parameters as a set of blobs. */  vector<shared_ptr<blob<dtype> > > blobs_;  /** Vector indicating whether to compute the diff from each param blob. */  vector<bool> param_propagate_down_;  /** the vector, indicates whether each top blob have a non-zero weight in   * The  objective function. */  Vect Or<dtype> Loss_;

In the source code can be seen, there is indeed a vector to store a lot of blob variable blobs_;

Param_propagate_down_ in the comments, indicating whether the blobs in this layer need to be computed diff; Why do you need this thing? Here we need to emphasize again: net > Layer > BLOB. A layer may contain multiple blobs, such as multiple bottom, and multiple top.

So what does Loss_ do? I can't see it for the time being.

As for the layerparameter definition in Caffe.proto:

Message Layerparameter {repeated string bottom = 2;//The name of the bottom blobs repeated string top = 3;//The NAM E of the top blobs Optional String name = 4; The layer name//Rules controlling whether and when a layer was included in the network,//based on the current NetS  Tate.  You may specify a non-zero number of the rules//to include OR exclude, and not both.  If no include or exclude rules be//specified, the layer is always included.  If the current netstate meets//any (i.e., one or more) of the specified rules, the layer is//included/excluded.  Repeated netstaterule include = 32;  Repeated netstaterule exclude = 33; NOTE//ADD new layertypes to the enum below in lexicographical order (all than//starting with NONE), starting WI Th the next available ID in the comment//line above the enum.  Update the next available ID when you add a new//Layertype. Layertype Next available id:38 (last Added:contrastive_loss) enum Layertype {// "NONE" layer type is a 0th enum element so the We don ' t cause confusion//by defaulting to an existent layertype (inst    EAD, should usually error if//the type is unspecified).    NONE = 0;    Absval = 35;    accuracy = 1;    Argmax = 30;    BNLL = 2;    CONCAT = 3;    Contrastive_loss = 37;    convolution = 4;    DATA = 5;    dropout = 6;    Dummy_data = 32;    Euclidean_loss = 7;    Eltwise = 25;    FLATTEN = 8;    Hdf5_data = 9;    Hdf5_output = 10;    Hinge_loss = 28;    Im2col = 11;    Image_data = 12;    Infogain_loss = 13;    Inner_product = 14;    LRN = 15;    Memory_data = 29;    Multinomial_logistic_loss = 16;    MVN = 34;    POOLING = 17;    POWER = 26;    RELU = 18;    SIGMOID = 19;    Sigmoid_cross_entropy_loss = 27;    Silence = 36;    SOFTMAX = 20;    Softmax_loss = 21;    SPLIT = 22;    SLICE = 33;    TANH = 23;    Window_data = 24;  THRESHOLD = 31; } Optional Layertype type = 5; The layer type from the enum above//The blobs containing the numeric parameTers of the layer repeated blobproto blobs = 6;  The names of the parameter BLOBs--useful for sharing parameters among//layers (but never required).  Repeated string param = 1001; Whether to require GKFX weights to has the same shape, or just the same//count--defaults to STRICT if Unspecifi  Ed.  Repeated dimcheckmode blob_share_mode = 1002; Enum Dimcheckmode {//neil:disabled for Windows////STRICT (default) requires that num, channels, height, width eac  H match.    STRICT = 0;    PERMISSIVE requires only the count (Num*channels*height*width) to match.  PERMISSIVE = 1; }//The ratio that's multiplied on the global learning rate.  If you want to//set the learning ratio for one blob, you need to set it for all blobs.  Repeated float BLOBS_LR = 7;  The weight decay that's multiplied on the global weight decay.  Repeated float Weight_decay = 8;  The amount of weight to assign each top blob in the objective. Each layer assigns a default ValUE, usually of either 0 or 1,//to each top blob.  Repeated float loss_weight = 35;  Optional Accuracyparameter Accuracy_param = 27;  Optional Argmaxparameter Argmax_param = 23;  Optional Concatparameter Concat_param = 9;  Optional Contrastivelossparameter Contrastive_loss_param = 40;  Optional Convolutionparameter Convolution_param = 10;  Optional Dataparameter Data_param = 11;  Optional Dropoutparameter Dropout_param = 12;  Optional Dummydataparameter Dummy_data_param = 26;  Optional Eltwiseparameter Eltwise_param = 24;  Optional Hdf5dataparameter Hdf5_data_param = 13;  Optional Hdf5outputparameter Hdf5_output_param = 14;  Optional Hingelossparameter hinge_loss_param = 29;  Optional Imagedataparameter Image_data_param = 15;  Optional Infogainlossparameter Infogain_loss_param = 16;  Optional Innerproductparameter Inner_product_param = 17;  Optional Lrnparameter Lrn_param = 18;  Optional Memorydataparameter Memory_data_param = 22;  Optional Mvnparameter Mvn_param = 34; Optional PoolIngparameter Pooling_param = 19;  Optional Powerparameter power_param = 21;  Optional Reluparameter Relu_param = 30;  Optional Sigmoidparameter Sigmoid_param = 38;  Optional Softmaxparameter Softmax_param = 39;  Optional Sliceparameter Slice_param = 31;  Optional Tanhparameter Tanh_param = 37;  Optional Thresholdparameter Threshold_param = 25;  Optional Windowdataparameter Window_data_param = 20;  Parameters for Data pre-processing.  Optional Transformationparameter Transform_param = 36; Note:certain layers May has more than than one computational engine//for their implementation.  These layers include an engine type and//engine parameter for selecting the implementation.  The default for the engine was set by the engine switch at Compile-time.  Deprecated:the layer parameters specified as a v0layerparameter.  This should never is used by any code except to upgrade to the new/Layerparameter specification. Optional V0layerparameter layer = 1;}

The equivalent is the belief description of each parameter in the layer, such as the name of the layer, what the next layer of the layer is, and the learning rate and so on.

Continue to see HPP will find a variety of virtual functions, first look at protected inside the method:

2 Feedforward, feedback function:

  /** @brief Using The CPU device, compute the layer output. */virtual void forward_cpu (const vector<blob<dtype>*>& Bottom, vector<blob<dtype>*>* to  p) = 0;   /** * @brief Using the GPU device, compute the layer output.   * Fall back to FORWARD_CPU () if unavailable. */virtual void Forward_gpu (const vector<blob<dtype>*>& Bottom, vector<blob<dtype>*>* to    p) {//LOG (WARNING) << "Using CPU Code as backup.";  Return forward_cpu (bottom, top); }/** * @brief Using the CPU device, compute the gradients for any parameters and * for the bottom blobs if PR   Opagate_down is true. */virtual void backward_cpu (const vector<blob<dtype>*>& Top, const vector<bool>& Propagat  E_down, vector<blob<dtype>*>* bottom) = 0; /** * @brief Using the GPU device, compute the gradients for all parameters and * for the bottom blobs if Propa Gate_down is True.   * Fall back to BACKWARD_CPU () if unavailable. */virtual void Backward_gpu (const vector<blob<dtype>*>& Top, const vector<bool>& Propagat    E_down, vector<blob<dtype>*>* bottom) {//LOG (WARNING) << "Using CPU Code as backup.";  BACKWARD_CPU (top, propagate_down, bottom); }

These functions should be easy to understand. The point is that several xx_gpu (), why the inside call is XX_CPU () Ah?!! If the call XX_CPU () is understood when the GPU is not available, but nothing has been done to call XX_CPU directly? But this is virtual function ha, do not worry, in the detailed implementation should be able to understand why.

3 Check that the number of blobs is correct:

  /** * Called by the parent Layer's SetUp to check the number of bottom * and top Blobs provided as input match   The expected numbers specified by * The {exactnum,min,max}{bottom,top}blobs () functions. */virtual void checkblobcounts (const vector<blob<dtype>*>& Bottom, const v ector<blob<dtype>*>& top) {if (exactnumbottomblobs () >= 0) {check_eq (Exactnumbottomblobs (), b Ottom.size ()) << type_name () << "Layer takes" << exactnumbottomblobs () << "Bo    Ttom Blob (s) as input. ";} if (minbottomblobs () >= 0) {Check_le (Minbottomblobs (), Bottom.size ()) << type_name () << "La    Yer takes at least "<< minbottomblobs () <<" bottom blob (s) as input. ";} if (maxbottomblobs () >= 0) {Check_ge (Maxbottomblobs (), Bottom.size ()) << type_name () << "La Yer takes at the most "<< MAXBOTTOMBLOBS () << "bottom blob (s) as input."; } if (Exactnumtopblobs () >= 0) {check_eq (Exactnumtopblobs (), Top.size ()) << type_name () <&lt ;    "Layer produces" << exactnumtopblobs () << "top blob (s) as output.";} if (mintopblobs () >= 0) {Check_le (Mintopblobs (), Top.size ()) << type_name () << "Layer Produ    Ces at least "<< mintopblobs () <<" top blob (s) as output. ";} if (maxtopblobs () >= 0) {Check_ge (Maxtopblobs (), Top.size ()) << type_name () << "Layer Produ    Ces at most "<< maxtopblobs () <<" top blob (s) as output. ";} if (Equalnumbottomtopblobs ()) {check_eq (Bottom.size (), Top.size ()) << type_name () << "Layer PR Oduces one top blob as output for each "<<" bottom blob input. ";}}

Here are some strange functions, such as: Exactnumbottomblobs (), but do not affect reading, checkblobcounts () The function is to detect the input bottom blob and output of the top blob is within the specified range, Since it is a specified range, what would naturally be the place to designate this range? Continue to read the source with doubt, somewhere, always understand.

4 last protected function,setlossweight ():

/** * Called By SetUp to initialize the weights associated with a top blobs in * the loss function.   Store Non-zero loss weights in the diff blob. */inline void setlossweights (vector<blob<dtype>*>* top) {const int num_loss_weights = Layer_param_.loss_    Weight_size (); if (num_loss_weights) {check_eq (Top->size (), num_loss_weights) << "Loss_weight must be" "Unspecif      IED or specified once per top blob. "; for (int top_id = 0; top_id < top->size (); ++top_id) {const Dtype loss_weight = Layer_param_.loss_weight (top        _ID);        if (Loss_weight = = Dtype (0)) {continue;}        This->set_loss (top_id, loss_weight);        const int count = (*top) [Top_id]->count ();        dtype* loss_multiplier = (*top) [Top_id]->mutable_cpu_diff ();      Caffe_set (count, Loss_weight, loss_multiplier); }    }  }

Here is the setting of the loss value, usually only in the loss function of the layer to calculate the loss value, so this function is mainly used in the last layer. Probably the equivalent of copying the value of the loss, so you can make a bold guess, this is the first step of feedback.

Caffe Source Reading 4-layer.hpp