Gradient Descent <1>

Minimum: gradient descent; maximum: Gradient Descent

(1) Batch Gradient Descent-minimizes the loss function of all training samples, so that the final solution is the global optimal solution, that is, the parameter to be solved is to minimize the risk function.

(2) random gradient descent-minimizes the loss function of each sample. Although not the loss function obtained by each iteration is in the global optimal direction, however, the larger overall direction is to the global optimal solution, and the final result is usually near the global optimal solution.

 

Let theta (j) fall in the fastest direction along the ladder, batch gradient descent to the multi-sample gradient de-mean;

Gradient Descent:

Http://blog.csdn.net/acdreamers/article/details/27660519

Related Expressions:

Batch Gradient Descent:

Http://blog.csdn.net/lilyth_lilyth/article/details/8973972

Related Expressions:

 

Gradient Descent <1>