CNN Softmax regression BP derivative

Content from UFLDL, code reference from Tornadomeet CnnCost.m1.Forward propagation

Convolvedfeatures = Cnnconvolve (Filterdim, numfilters, images, Wc, BC); %for the first arrow activationspooled= Cnnpool (Pooldim, convolvedfeatures);corresponds to a second arrow%corresponds to the 3rd arrow, that is, the flat roll activationspooled=reshape (activationspooled,[],numimages);%the probability of the beginning of the calculation of Softmax belongs to various probs=zeros (numclasses,numimages);%wd=(numclasses,hiddensize), each column of the probs represents an output%m=wd*ah+BdM= Wd*activationspooled+repmat (bd,[1, Numimages]); % This step can be omitted, the reason can be done is exp (a+b) =exp (a) exp (b) M= Bsxfun (@minus, M,max (m,[],1));%m=exp (wd*ah+BD) M=exp (M);%Normalizeprobs= Bsxfun (@rdivide, M, sum (m));

2.Back Propagation

% first need to get labels into one-Hot Coding%Igroundtruth in the corresponding picture= Full (sparse (labels, 1:numimages, 1));%p-Idelta_d=-(groundtruth-probs); %ah (P-I), where the derivative of the normal term is appended to the Wd_grad= (1./numimages) *delta_d*activationspooled'+lambda*wd;Bd_grad = (1./numimages) *sum (delta_d,2); %Note Here are the requirements and%The J to the right of reshape in the corresponding graph delta_s= Wd'*delta_d;delta_s=reshape (delta_s,outputdim,outputdim,numfilters,numimages);% corresponds en route 1/4, each component of the delta_s is expanded to 4 forI=1: Numimages forJ=1: Numfilters Delta_c (:,:, J,i)= (1./pooldim^2) *Kron (Squeeze (delta_s (:,:, J,i)), ones (Pooldim)); EndEnd%for the lower left, but at this point ximage has not been delta_c= convolvedfeatures.* (1-convolvedfeatures). *Delta_c; forI=1: Numfilters wc_i=zeros (Filterdim,filterdim);  forJ=1: Numimages%here conv2 very ingenious wc_i= Wc_i+conv2 (Squeeze (Images (:,:, J)), Rot90 (Squeeze (Delta_c (:,:, I,j)), 2),'valid'); End% Wc_i = CONVN (images,rot180 (Delta_c (:,:, I,:))),'valid'); %Add penalize Wc_grad (:,:, i)= (1./numimages) *wc_i+Lambda*Wc (:,:, i); Bc_i=Delta_c (:,:, I,:); Bc_i=bc_i (:); Bc_grad (i)= SUM (bc_i)/Numimages;end

The correctness of the above conv2 can be verified by the following method

A=rand (9,9); B=rand (3,3); C1=conv2 (A, B,'valid'); B=zeros (3);  for I=1:7 for     j=1:7        B=b+ (A (i:i+2,j:j+2) *C1 (I,J));    EndEnd% See B and conv2 result same conv2 (A,rot90 (C1,2),'valid') b

CNN Softmax regression BP derivative