Deep convolutional neural network based on Theano

1. Introduction

convolutional Neural Networks (convolutional neural Networks, CNN) are sensitive to only parts of the field of vision that are affected by cells on the retina, a part of which is known as the sensation domain (receptive field ). convolutional neural Networks Use this mechanism, where each neuron is connected to only a subset of the inputs.

2. Sparse Connections

CNNs reveals local correlations in space by means of local connections. The input of the hidden unit in the $m $ layer is derived from the weighted sum of a portion of the element in the $m -1$ layer, which is spatially contiguous in the sensing domain. Such as:

The $m -1$ layer can be imagined as a retinal input. The cells of the $m $ layer have a width of 3, so they are only connected to 3 adjacent neurons in the retinal layer. This is also true $m the +1$ layer's elements are connected to the layer below. Each neuron does not respond to changes that are not within the sensing domain, so the structure above ensures that a "filter" is learned to produce a strong response to the input pattern of the local space.

However, as shown in the figure above, many of these filter layers are layered, the local perception gradually becomes global awareness, $m each unit of the layer is only part of the input perception, and $m +1$ layer of the cells will $m the layer of the perceptual results to form a full sense of the input layer, so $m+1$ The hidden layer element can be regarded as a non-linear encoding of a feature with a width of 5.

3. Sharing weights (sharedWeights)

In the CNNs, each filter $h _{i}$ progressively across the input layer over and over again. Duplicate unit sharing parameters (weight vector and bias), thus forming a feature map.

In, 3 hidden layer units belong to the same feature map, the same color weight value is shared, that is, equal.

In this way, filters can be detected in any position in the visual layer of the image, and weight sharing greatly reduces the number of parameters that need to be learned.

4. Details and symbols

A feature map is obtained by repeatedly applying a function to the entire sub-region of the image, i.e., a linear filter is used to convolution the image, plus a biased term, followed by a nonlinear function. If you use $h ^{k}$ to represent the feature map of the $k $ layer, the corresponding filter is determined by the $W ^{k}$ and bias $b _{k}$, then the feature map $h ^{k}$ can be computed from the next (using Tanh for nonlinear functions):

$h _{ij}^{k}=tanh (w^{k}*x) _{ij}+b_{k}$

In order to get a richer representation of the data, each hidden layer is usually composed of multiple feature graphs: $\{h^{\text{(k)}},k=0,... k\}$. The weight $W $ is represented by a 4-dimensional tensor, and 4 of the dimensions represent: The objective feature graph, the source feature graph, the vertical coordinates of the source feature graph, and the horizontal coordinates of the source feature graph. The offset $b $ is represented by a vector, where each element is the offset of each target feature graph. Can be expressed as follows:

In the $W _{ij}^{kl}$ represents a connection weight between the $m the -1$ layer of the $k $ per pixel of the feature map and the $m $ (i,j) $ for the $l $ of the map in the section of the $/a feature graph.

Deep convolutional neural network based on Theano