Andrew ng Machine Learning Introductory Learning Note (iv) neural Network (ii)

This paper mainly records the cost function of neural network, the usage of gradient descent in neural network, the reverse propagation, the gradient test, the stochastic initialization and other theories, and attaches the MATLAB code and comments of the relevant parts of the course work.

Concepts of neural networks, models, and calculation of predictive classification using forward propagation refer to Andrew Ng Machine Learning Introductory Note (iv) neural Network (i)

http://blog.csdn.net/scut_arucee/article/details/50144225 . Neural network Solving classification problem model parameters

M-M group training data (x (1), Y (1)), (x (2), Y (2)), and, (X (m), Y (M)) (x^{(1)},y^{(1)}), (x^{(2)},y^{(2)}), \cdots, (x^{(m)},y^{(M)});

The total number of layers of the neural network l l;

Number of elements in the L-L layer Sl s_l (excluding deviation units);

The number of cells in the output layer is K K.

① for two kinds of classification problems

Y=0 or 1 y=0 or 1, only one output unit, hθ (x) ∈r H_\theta (x) \in\mathbb{r}, so sl=1 s_l=1, that is k=1 k=1.

② for multi-class classification problems

Y y is a vector, y∈rk,hθ (x) ∈rk,sl=k (k⩾3) Y\in\mathbb{r}^k,h_\theta (x) \in\mathbb{r}^k,s_l=k (K\GEQSLANT3). two. Cost function of neural networks

1. The cost function of regularization logistic regression
J (θ) =−1m[∑i=1m