The algorithm of flowing water, the loss function of/mle

Machine learning algorithm can be said that many, if rote, can only remember the derivation process and steps, after a period of time can not remember, only vaguely remembered some shadow. Therefore, we should find some general methods of the algorithm to understand the idea of the algorithm and the derivation process.

I think that the maximum likelihood estimation and loss function, which is the universal frame of machine learning algorithm, is the key to mastering the machine learning algorithm.

Below, use the actual algorithm to confirm the power of the two keys.

1.Linear Regression. Can be translated into the demand

The loss function is minimized to solve the parameter θ.

Then, the gradient descent method is used to solve the θ effectively. Besides the gradient descent tool, there are some important tools of quasi-Newton method and Lagrange multiplier method.

2.Logistic Regression. With the model

, the maximum likelihood function is obtained first:

Then we take the logarithm of this likelihood function and get

　

In practice, a minus sign is usually added to the front, and the minimum is changed.

3.SVM

The algorithm of flowing water, the loss function of/mle