Minimalist notes: Dynamic Routing Between Capsule

Source: Internet
Author: User
Minimalist notes: Dynamic Routing Between Capsule

Capsule The first bullet, is a capsule simple demo, in order to prove that capsule can be used.

If the traditional neural network neurons are scalar neurons, capsule can be understood as tensor neurons (2 dim of Spatial & 1 Dim of feature vector). Vector length is the probability of an object appearing, and orientation is an object attribute.

Use the three-tier structure test on Mnist:
Image–conv1–primarycaps–digitcaps

Between the first two layers are normal convolution, primarycaps equivalent to the convolution Layer packaging components capsule (calculation of the essence and convolution exactly the same, only divided into a group), Primary-digit is the first capsule in each vector (6*6*32, 8- Dim) to 10 16-dim output vectors do a full join, each edge of the weight is the matrix Wij W_{ij} (size:8*16) Get U^ij \hat{u}_{ij} (6*6*32*10), then weighted sum and 10 vector SJ S_ The length of J is done nonlinearly normalized into the output vector VJ V_j, wherein the weighted CIJ C_{ij} is the true dynamic routing mode update part, CIJ=EXP (bij)/∑kexp (Bik) C_{ij} = exp (B_{ij})/\sum_k E XP (B_{ik}), that is, softmax operation, update bij=bij+u^tijvj B_{ij} = b_{ij} + \hat{u}_{ij}^t V_j

Because the dynamic routing this part of the argument is a little bit more than the full connection (many of which are the C), so definitely need to improve.

Are all connected to highlight affine transformation robust is not necessary, but it is noteworthy that the last obtained 10 VJ V_j through three-layer full connection to the original image, This error plus training can improve the accuracy of a certain degree (good features indicate a strong ability to reconstruct ...). )

Paper Address: https://arxiv.org/abs/1710.09829

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.