cs231n 2017 Learning Notes 04--reverse propagation with neural network backpropagation and neural Networks

This blog content from Stanford University cs231n lecture 4-backpropagation and neural Networks

Course Website: http://cs231n.stanford.edu/syllabus.html

From the course website can find more detailed information, to view the video needs Fq on YouTube, if not FQ or feel more trouble, can also be downloaded from the Baidu Cloud link I gave.

Course Videos & Handouts Download: Http://pan.baidu.com/s/1gfu51KJ

Calculation Diagram Computational graphs

The calculation diagram is the complex formula is expressed graphically, very intuitive.

Reverse propagation backpropagation:

The inverse propagation algorithm takes advantage of the chain rule of derivation, starting from the result, the derivative of each variable is introduced step by step, which is also the origin of the reverse propagation name.

The diagram shows the most simple node, of course, can also be appropriate to combine several nodes to form a new slightly more complex node. This is in itself a kind of trade-off, if you want to do less calculus, you have to use more simple nodes, but this will cause the whole calculation diagram is huge, bloated.

Characteristics of common nodes

cs231n 2017 Learning Notes 04--reverse propagation with neural network backpropagation and neural Networks