[Machine learning] Logistic regression, logistic regression | classification, classification

This is the study note of Andrew Ng's public course on machine learning.
Examples of reality are spam/non-spam, tumors are benign or malignant, and so on.
How to classify. I have accumulated an experience from high school mathematics. Assuming that the linear equation is f (x) = 0, then the point to the left of the line is taken to the left of the linear equation, resulting in the result < 0; instead, the point to the right of the line is brought into the left side of the line equation, resulting in > 0.
So, if we can find such a line, so that its left point belongs to Class A, the right point belongs to Class B, then there is a new point to come in, we can know what kind of this point belongs to.
Yes, or a straight line (here, for the sake of explanation, it's a simple model.) It can also be multiple feature, Gaozi and so on. ）




Why is it that regression?  Because we used the hypothesis of linear regression. Why is it that it is logistic. Because the results here are only 0 and 1 (discrete). The result of linear regression is continuous.
Above only with > or < expression of the formula, no this method to find costfunction, can not seek the optimal solution. So the great scientist thought of the following formula, and made a conversion to hypothesis.

You can see that after the conversion, the range of H (x) is 0~1,z >0, h (x) > 0.5. Z < 0 o'clock, H (x) < 0.5. So we can judge the classification by the value of H (x).

Ask Theta, of course, still rely on min costfunction because h (x) changes, costfunction if the form of linear regression, it is not convex, you can not use gradient descent method convex And the comparison of Non-convex

So, the predecessor thought to give H (x) plus log way as costfunction, such a costfunction into convex



From the above two curves, when the y=1, Costfunction gradually reduce the method is to make H (x) toward 1 when the y=0, to costfunction gradually reduced, is to make H (x) toward 0, which integrates our meaning.
The two formulas are not good at all, so the following combination, and the derivation after the formula is so harmonious. Beauty.

Using the gradient descent method to obtain the updated theta, the expression can be similar to the linear regression, the difference is only on the H (X).
In fact, from the value of H (x), we can understand that the probability of predicting the result of 1 is H (x)

Multi-Class classifiation One-vs-all Classification. One-to-many classifications, that is, to convert multiple categories into 2 classes, Class A, and all the remaining classes. This allows the use of two types of logistic regression methods.
By means of One-vs-all, a hypothesis is trained for each class. Finally, by comparing the results of multiple hypothesis and then predicting