Stanford Wunda-cousera Course notes-logistic regression _ machine learning

CSDN blog first, yards of hard, I hope to help you

Logistic regression is a widely used classification algorithm, this paper discusses two classification problems, for multiple classification can be done through a pair of more than two classification calculation,

You can also reconstruct the taxonomy model.

1, the use of logistic regression motivation:

1 avoid the interference of special samples

Through the subordinate mate graph, it can be clearly seen that because of the existence of special samples at the right end, the prediction line has a large deviation, and may not be the actual discrepancy.

2 so that the value of the predictive function falls in a reasonable interval

Using the assumption of linear regression, the output value of the assumed function is not on [0,1], and logistic regression can more reasonably describe the possibility that each sample belongs to each category.

2. Representation of hypothetical functions

The assumption of linear regression is:

The logistic regression is assumed to be:

where G (z) is the sigmoid function, the predictive function measures the probability of a positive sample for the sample.

The sigmoid function has the following properties:

If the threshold value is 0.5 of the sigmoid function, the

The predicted value of the sample is a positive sample, otherwise it is a negative sample. The x here can be understood as a feature, such as a polynomial feature, or a characteristic that has practical significance.

At the same time, the so-called decision boundary is determined by the parameters fitted to the training sample.

3. Optimization function

Optimization objective function of linear regression:

But for the Logisti regression, this objective function can not be used to get the parameters. Because it is no longer a linear convex function, if the optimization algorithm is used

When solving a parameter, the global minimum is not obtained. For this reason, the optimal objective function of logistic regression is:

First, this is a convex function, the surrogate definition, and the cost function for y = 1:

According to the optimization theory, this is a convex function.

Secondly, the intuitive understanding of this cost function is: when y = 1 o'clock, the nearer 1, the lower the cost, the closer to 0, the greater the cost.

The cost function can be written in the following general form:

4. Parameter solving algorithm-gradient descent method

The cost function or optimization objective function of the model is:

Gradient Descent algorithm process:

The iteration substance of the parameter is performed in the following style:

The difference between this and linear regression is only the difference of the hypothetical function, which embodies the high consistency.

5.regularized Logistic regression

When the number of features is too large, assuming the function is too complex, if the number of samples is not enough, it is easy to have a fitting problem.

The improved approach is to add the following for the cost function:

The parameters that correspond to each feature.

This technique, which is similar to the penalty number method, reduces the complexity of the hypothetical function, and the iterative process of the parameters is as follows:

This distinguishes the constant feature from the fact that it has no effect on the complexity of the assumed function because it only translates the hypothetical function.

The selection here is very important, when too large, will lead to regularization, too small, and can not play a role.