NG Machine Learning Video Notes (ii)--gradient descent algorithm interpretation and solving θ

NG Machine Learning Video notes (ii)

--Gradient descent algorithm interpretation and solving θ

(Reproduced please attach this article link--linhxx)

First, the interpretation gradient algorithm

A gradient algorithm formula and a simplified cost function diagram, as shown in.

1) Partial derivative

By the know, at point A, its partial derivative is less than 0, so θ minus less than 0 of the number, equivalent to add a number. In addition, it can be seen from the figure that at point A is not the best point, you need to continue to the right, that is, a needs to increase. Therefore meet the requirements.

For at point B, you can get the same result that needs to be reduced.

2) Learning rate α

α indicates the rate at which the point moves toward the minimum point, and the alpha value needs attention.

The value is too large, the distance of each move is too long, may result in near the minimum point, the movement will exceed the minimum point of the position, resulting in constant in greater than, less than the minimum point of the position offset, unable to converge;

The value is too small to move very slowly, which can cause the program to take too long.

In addition, because the closer to the minimum point, the value of the partial derivative (the absolute number) is smaller, so the rate of change itself becomes slow, so after the alpha is selected, it does not need to adjust the value, its own slow rate.

Second, gradient algorithm defects

It is known that for the cost function with multiple minimum points, the gradient algorithm can only take the local minimum point, that is, the minimum point of the function, but there is no guarantee that the point is the minimum value point.

Three, the solution θ

The formula, as shown, is essentially the result of a partial fall.

Constant calculation of θ0 and θ1, until the partial derivative is 0 (or set less than a certain threshold), then stop the calculation, at this time the result is a local optimal result for a certain starting point.

--written by Linhxx

More recent articles, please pay attention to the public number "machine learning", or scan the right QR code.

NG Machine Learning Video Notes (ii)--gradient descent algorithm interpretation and solving θ