Andrew ng Machine Learning (ii): Logistic regression

1. What is the resolution of logistic regression?

Logistic regression is used for classification problems.

For the two classification problem, enter multiple features and the output is yes or no (you can also write 1 or 0).

Logistic regression is one such model for classification.

2. What is logistic regression?

The logistic regression is based on the linear regression.

First, linear regression maps multiple features to a variable.

After that, set a threshold on the variable. Greater than this threshold is judged to be, output 1; Less than this threshold value is no, output 0.

can also not output 1 or 0, and the output is 1 probability, and then whether the probability is greater than 0.5 to judge yes or No.

Logistic regression is a model where the output is a probability of 1 (the sigmoid function):

For ΘTX, the threshold value for determining whether the output is 1 is θtx=0.

3.Decision boundary

Assuming we have found a suitable logistic regression model, it is equivalent to determining a set of θ so that the θtx=0 is the appropriate threshold value. At this point, Θtx=0 will split the feature space, called decision boundary.

In fact, decision boundary is another description of the post-training model.

4. Cost Function for logistic regression

In order to evaluate the degree of model fit, the cost function of linear regression is not applicable to logistic regression, which is analogous to linear regression.

There is a need to find a way to judge deviations.

For a single sample, consider the logarithmic function:

When it is actually 1 o'clock, the probability of 1 is greater (H stands for probability), the smaller the deviation. Similarly, when the actual is 0 o'clock, the probability of 1 is greater, the greater the deviation.

Sum of deviations of all samples, cost function can be written in the following formula

5. Training model (find the parameter with the least deviation)

The simplest is also the gradient drop. The formula is written once more:

The detailed derivation and the partial derivative can be seen: an explanation.

In addition, it can be used conjugate gradient, BFGS, L-BFGS and other algorithms to calculate parameters, more efficient, but more complex, not easy to debug.

For the encapsulated algorithm, only the cost function and the partial derivative can be supplied, and the α and the calculation are automatically selected.

6.matlab implements a logistic regression.

%data.csv [Feature1 Feature2 y]

Sample = Csvread (' data.csv ');
X (:, 2) = sample (:, 1);
X (:, 3) = sample (:, 2);
Y (:, 1) = sample (:, 3);
X (:, 1) = ones (Size (X (:, 2)));

m = Length (X (:, 2));

%choose Logistic regression as model
%start with

theta = [1;2;1];
Alpha = 1;
Times = 5000;% iterations
J_theta = [1:times];

%cost function J_theta & gridient gri_j using Gridient descent

For I=1:times
H = 1./(1+exp (-x*theta));
J_theta (i) = ( -1/m) * (log (H ') *y+log (1-h ') * (1-y));
Gri_j = (1/m). * ((H-Y) ' * x) ';
theta = Theta-alpha.* Gri_j;
End

Plot (J_theta);
% deci_bond_y =-(1/theta (3)). * (Theta (1) +theta (2). * X (:, 2));
%
% Stem (x (:, 2), X (:, 3), ' X ');
% hold on;
% Plot (x (:, 2), deci_bond_y);

7. Multivariate classification

For each category, consider this category and "other categories" (called one versus all in the course), which will change back to the two-dollar category.

Then we classify each category by two, and get N classifiers.

When testing is required, input the data into each classifier, selecting one of the largest probabilities as the output.

Summary

Logistic regression is built on the basis of linear regression. The model is: the probability that the output is 1 through the sigmoid function. The application should conform to the Bernoulli distribution in the output.

The gradient descent algorithm is also useful, and there are some more efficient algorithms. At first, you can use it first, then go deep into your study.

Linear regression and logistic regression are used to deal with different problems, but the methods are: analyzing the data, selecting the model, optimizing the data, selecting the algorithm, training, and getting the trained model.

Andrew ng Machine Learning (ii): Logistic regression