Minimalist notes: Dynamic Routing Between Capsule

Minimalist notes: Dynamic Routing Between Capsule

Capsule The first bullet, is a capsule simple demo, in order to prove that capsule can be used.

If the traditional neural network neurons are scalar neurons, capsule can be understood as tensor neurons (2 dim of Spatial & 1 Dim of feature vector). Vector length is the probability of an object appearing, and orientation is an object attribute.

Use the three-tier structure test on Mnist:
Image–conv1–primarycaps–digitcaps

Between the first two layers are normal convolution, primarycaps equivalent to the convolution Layer packaging components capsule (calculation of the essence and convolution exactly the same, only divided into a group), Primary-digit is the first capsule in each vector (6*6*32, 8- Dim) to 10 16-dim output vectors do a full join, each edge of the weight is the matrix Wij W_{ij} (size:8*16) Get U^ij \hat{u}_{ij} (6*6*32*10), then weighted sum and 10 vector SJ S_ The length of J is done nonlinearly normalized into the output vector VJ V_j, wherein the weighted CIJ C_{ij} is the true dynamic routing mode update part, CIJ=EXP (bij)/∑kexp (Bik) C_{ij} = exp (B_{ij})/\sum_k E XP (B_{ik}), that is, softmax operation, update bij=bij+u^tijvj B_{ij} = b_{ij} + \hat{u}_{ij}^t V_j

Because the dynamic routing this part of the argument is a little bit more than the full connection (many of which are the C), so definitely need to improve.

Are all connected to highlight affine transformation robust is not necessary, but it is noteworthy that the last obtained 10 VJ V_j through three-layer full connection to the original image, This error plus training can improve the accuracy of a certain degree (good features indicate a strong ability to reconstruct ...). ）

Paper Address: https://arxiv.org/abs/1710.09829