Perceptron and Neural network training (formula derivation and C + + implementation) __c++

Perception Device

The Perceptron is a two-dollar linear classifier whose goal is to find a hyperplane that separates the training samples (categories), which are expressed as

Generally, the activation function is not a step function, which is commonly used with the sigmoid function (its derivative: σ′=σ (1−σ) \sigma ' = \sigma (1-\sigma))

The cost function (least mean square error) on the training set of 1 samples is usually
C=12|a−y|2 C = \frac{1}{2} |a-y|^2
The new value of the W and b b of the Step (learning rate) for Α\alpha during the training process is
W′IB′=WI−Α∇WIC=B−Α∇BC \begin{align} w_i ' &= w_i-\alpha \nabla_{w_i}c\\ B ' &= B-\alpha \nabla_{b}C \end{align}
The gradient is calculated separately, because in the neuron, the output from the previous step is the next input, which is the independent variable in the next step, which can be obtained by the chain rule.
∂c∂wi∂c∂b=∂c∂z∂z∂wi=∂c∂z∂z∂b \begin{align} \frac{\partial c}{\partial w_i} &= \frac{\partial C}{\partial z} \frac{\ Partial z}{\partial w_i}\\ \frac{\partial c}{\partial B} &= \frac{\partial c}{\partial z} \frac{\partial z}{\partial b } \end{align}
which