NOTES: Cs231n+assignment2 (Assignment II) (iii)

Finally came to the end of the big Boss, convolutional neural Network ~

Here I would like to focus on the implementation of the code, specific CNN knowledge point to write a good write later, CNN's code is to add convolution layer and pool session layer.

I. Convolution layer

The forward propagation of the convolution layer is still relatively easy, our main concern is the reverse propagation, we can see that:

　　

defconv_forward_naive (x, W, B, Conv_param): stride, Pad= conv_param['Stride'], conv_param['Pad'] N, C, H, W=X.shape F, C, HH, WW=W.shape x_padded= Np.pad (x, ((0, 0), (0, 0), (pad, pad), (pad, pad)), mode='constant')#complement 0H_new = 1 + (H + 2 * pad-hh)/Stride W_new= 1 + (W + 2 * pad-ww)/Stride S=Stride Out=Np.zeros ((N, F, H_new, w_new)) forIinchXrange (N):#ith Image         forFinchXrange (F):#fth Filter             forJinchxrange (h_new): forKinchxrange (w_new): Out[i, F, J, K]= Np.sum (X_padded[i,:, J*s:hh+j*s, k*s:ww+k*s] * w[f]) + b[f]#corresponding phase multiplicationCache=(x, W, b, Conv_param)returnOut , Cachedefconv_backward_naive (dout, Cache): X, W, b, Conv_param=Cache Pad= conv_param['Pad'] Stride= conv_param['Stride'] F, C, HH, WW=W.shape N, C, H, W=X.shape h_new= 1 + (H + 2 * pad-hh)/Stride W_new= 1 + (W + 2 * pad-ww)/Stride DX=np.zeros_like (x) DW=Np.zeros_like (w) DB=Np.zeros_like (b) S=Stride x_padded= Np.pad (x, ((0, 0), (0, 0), (pad, pad), (pad, pad)),'constant') dx_padded= Np.pad (dx, (0, 0), (0, 0), (pad, pad), (pad, pad)),'constant')     forIinchXrange (N):#ith Image         forFinchXrange (F):#fth Filter             forJinchxrange (h_new): forKinchxrange (w_new): Window= X_padded[i,:, J*s:hh+j*s, k*s:ww+k*S] db[f]+=Dout[i, F, J, K] Dw[f]+ = window *Dout[i, F, J, K] Dx_padded[i,:, J*s:hh+j*s, k*s:ww+k*s] + = w[f] *Dout[i, F, J, K] #上面的式子, the key is the + number#UnpadDX = dx_padded[:,:, Pad:pad+h, pad:pad+W]returnDX, DW, DB

As mentioned in Http://www.cnblogs.com/tornadomeet/p/3468450.html, the BP algorithm of the convolution layer is calculated, that is, an orthodox convolution operation

Second, pooling layer

　　

defmax_pool_forward_naive (x, Pool_param): HH, WW= pool_param['Pool_height'], pool_param['Pool_width'] s= pool_param['Stride'] N, C, H, W=X.shape h_new= 1 + (H-HH)/s W_new= 1 + (W-WW)/s out=Np.zeros ((N, C, H_new, w_new)) forIinchxrange (N): forJinchxrange (C): forKinchxrange (h_new): forLinchxrange (w_new): Window= X[i, J, K*s:hh+k*s, l*s:ww+l*S] Out[i, J, K, L]=Np.max (window) Cache=(x, Pool_param)returnOut , Cachedefmax_pool_backward_naive (dout, Cache): X, Pool_param=Cache HH, WW= pool_param['Pool_height'], pool_param['Pool_width'] s= pool_param['Stride'] N, C, H, W=X.shape h_new= 1 + (H-HH)/s W_new= 1 + (W-WW)/S DX=np.zeros_like (x) forIinchxrange (N): forJinchxrange (C): forKinchxrange (h_new): forLinchxrange (w_new): Window= X[i, J, K*s:hh+k*s, l*s:ww+l*S] M=Np.max (window) #获得之前的那个值, so that as long as the windows==m will be able to get the corresponding position dx[i, J, K*s:hh+k*s, l*s:ww+l*s] = (window = = m) *Dout[i, J, K, L]returnDx

Third, the difference with the previous

Here the BN algorithm is not the same as before, because the network input becomes the Saptail

　　

defSpatial_batchnorm_forward (x, gamma, Beta, Bn_param): N, C, H, W=X.shape x_new= X.transpose (0, 2, 3, 1). Reshape (n*h*W, C) #分成不同的channel来算, so you can directly use the previous code out, the cache=Batchnorm_forward (x_new, gamma, beta, bn_param) out= Out.reshape (N, H, W, C). Transpose (0, 3, 1, 2)    returnOut , CachedefSpatial_batchnorm_backward (dout, Cache): N, C, H, W=Dout.shape dout_new= Dout.transpose (0, 2, 3, 1). Reshape (n*h*W, C) dx, Dgamma, Dbeta=Batchnorm_backward (dout_new, cache) dx= Dx.reshape (N, H, W, C). Transpose (0, 3, 1, 2)    returnDX, Dgamma, Dbeta

Iv. Summary

Assignment2 finally finished, in general. NumPy or more familiar, the specific operation should also be familiar with. The forward propagation of the convolution layer is very well understood, the reverse propagation and the previous difference is not big, just need to do a convolution operation.

NOTES: Cs231n+assignment2 (Assignment II) (iii)