An example of a step-by-step reverse communication

Description

Learning depth learning process, encountered an example to explain the reverse propagation algorithm Bowen, simple and rough easy to understand, very suitable for my small white, so I intend to translate a share, the English level is limited, translation of the wrong place please tell. The original address is here. Here is the translation. background

Reverse propagation is a commonly used algorithm in the training of neural networks. There are a lot of explanations on the back-propagation algorithm on the internet, but there are few real figures for you to illustrate. In this blog post, I will try to use a concrete example to explain how the reverse communication works, in order to determine whether they understand the correct, you can first of all according to their own understanding.
If this post attracts you, you can sign up for my newsletter, which will post some of the AI-related posts I'm working on. using reverse propagation in Python

You can also use my Python script to play a game, the program has implemented a reverse-propagation algorithm. Visualization of reverse propagation algorithm

To show an interactive visualization of neural network training, you can look at my neural network visualization. some of the other resources

If you think this tutorial is useful to you and want to continue learning about neural networks and their applications, I highly recommend Adrian Rosebrock tutorials, Getting started with Deep Learning and Python. General overview

For this tutorial, we're going to use two input nodes, two hidden nodes, and two output nodes. In addition, the output node and the hidden node will contain a bias. (This place, I feel should be input node and output node, but still according to the original translation.) ）
The following is a basic architecture:

For some numbers to count, the initialization weights, offsets, and the input and output of the training set are labeled below:

The goal of the reverse propagation algorithm is to optimize the weights so that the neural network can learn how to correctly map between any input and output.
The rest of the tutorial takes place around a single sample, given input times 0.05 and 0.10, and we want this neural network to correspond to output 0.01 and 0.99 respectively. go first, forward transmission.

First, let's look at what the neural network predicts at the current given value. So we feed the input data to the neural network first.
The total net input of each of our hidden layer nodes compresses the total net input with an activation function (the logical regression used here), and then repeats the process to the output node.

Total net access is also referred to as net input, with reference to certain data.

Here we calculate the total net input of H1 h_1

Then we use the logical regression function to obtain output H1 h_1 output:

Perform the same process for H2 h_2:

We will repeat this process for the nodes of the output layer, using the output of the hidden layer as input.
Here is the output O1 o_1:

and perform the same process on the O2 o_2:

calculation total error of outo2=0.772928465 out_{o2}=0.772928465

We can now use the square error function to compute the error of each output node and add them together to get the total error.

Etotal=∑12 (target−output) 2 e_{total}= \sum\frac{1}{2} (target-output) ^2

Some data call the target value the ideal value, the output value is called the actual value.

For example, the target output of the O1 o_1 is 0.01, but the output of the neural network is 0.75136507, so the error is:

Eo1=12 (targeto1−outo1) 2=12 (0.01−0.75136507) 2=0.274811083 e_{o1}=\frac{1}{2} (Target_{o1}-out_{o1}) ^2=\frac{1}{2} (0.01-0.75136507) ^2=0.274811083

Repeat this process for O2 O_2 (remember that the target value is 0.99),

Eo