Logistic regression & Recursive descent algorithm

0) The purpose of the recursive descent algorithm is to approximate the minimum value of the function by continually iterating, thus finding the parameter 1) the logistic regression is actually a classifier, using the existing sample to train the Sigmoid function.

(1) The general form of the sigmoid function:

(2) Graph of the sigmoid function:

(3) Prediction function:

For example, there is a sample X, he has 10 features:, according to the value of their predictive function can be obtained:

Then we can know the attribution of sample x: It is a class, otherwise it is another kind.

Note: This assumes a linear boundary condition: it is shaped like, not this. And the derivation is based on this hypothesis.

3) Derivation process

(1) First notice that the function domain is located in [0,1], the category is divided into 0, 12 classes.

The closer to 1, the more likely the sample belongs to Category 1; otherwise, the farther away from 1, the greater the likelihood that the sample will be in category 0.

So it can be seen as the features of a given sample X, and the known parameter θ, which belongs to the probability of Class 1:

, where the attribution category of the sample is represented by Y.

(2) Y satisfies two distributions

　　　　

　　　　

(3) According to MLE

　　　　

　　　　

(3) Define a new function:

　　　　

　　　　

(4) Derivation, to find the deviation of J (θ) to feature J

　　　　

Define A, and B

　　　　

　　　　

So:

　　　　

(5) Derivation of a, b

　　　　

　　　　

　　　　

　　　　

　　　　

　　　　

　　　　

　　　　

　　　　

　　　　

　　　　

　　　　

(6) Finally

　　　　

　　　　

　　　　

　　　　

　　　　

　　　　

Logistic regression & Recursive descent algorithm