Numerical Optimization-Newton method

Ng's Courseware 1 also speaks of Newton's law, and its corresponding update rule is

??

??

??

??

??

H corresponds to Hessian matrix

Http://en.wikipedia.org/wiki/Hessian_matrix

??

http://aria42.com/blog/2014/12/understanding-lbfgs/

Gives a more detailed explanation of Newton's Law:

??

Do the function Taylor expand http://en.wikipedia.org/wiki/Taylor_series

Compare a single-variable function Taylor expansion formula:

??

??

Consider that we need to find a position where the first derivative is 0.

??

Consistent with the NG courseware.

??

Newton's method and gradient descent are faster than convergence, but the computation of a larger (n+1,n+1) dimension of the Hessian matrix is calculated for each iteration, but if n is not particularly large then the overall view of Newton's method is quicker.

? ?

Numerical Optimization-Newton method