Deep Learning: 4 (Logistic regression exercises)

Deep Learning: 4 (Logistic Regression exercise)-tornadomeet-blog

Deep Learning: 4 (Logistic regression exercises)

Preface:

This section to practice the logistic regression related content, reference for web pages: http://openclassroom.stanford.edu/MainFolder/DocumentPage.php? Course = deeplearning & Doc = exercises/ex4/ex4.html. The training sample is characterized by the scores of 80 students in two courses. The sample value indicates whether the students are allowed to go to college. If so, it is expressed as '1, otherwise, it cannot be expressed as '0'. This is a typical binary classification problem. In this case, each of the 80 positive and negative samples is 40. This section uses logistic regression to solve the problem. The result after the solution is actually a probability value. Of course, by comparing with 0.5, it can become a binary classification problem.

Lab basics:

In the logistic regression problem, the logistic function expression is as follows:

The advantage of doing so is that the output results can be compressed to 0 ~ Between 1. The loss function in logistic regression is different from the loss function in linear regression:

If the Newton method is used to solve the parameters in the regression equation, the iteration formula of the parameters is:

The expression of the first-order derivative function and Hessian matrix is as follows:

Of course, to avoid using the for loop during programming, we should directly use the vector expressions of these formulas (for details, seeProgramContent ).

Some MATLAB functions:

Find:

Is a vector found. The result is the subscript Number of the value when the brace value of the find function is true.

Inline:

Construct an embedded function, similar to the mathematical derivation formula we wrote on the draft paper. Parameters are generally enclosed in single quotes, which are the function expressions. If multiple parameters exist, they are separated by single quotes. For example, if G = inline ('sin (alpha * X) ', 'x', 'alpha'), the binary function is g (x, alpha) = sin (alpha * X ).

Experiment results:

Training sample distribution chart and the learned classification interface curve:

Curve between the loss function value and the number of iterations:

Final output result:

It can be seen that when a child's first homework is 20 points and the second homework is 80 points, the probability that the child cannot go to college is 0.6680. Therefore, if the child is used as a binary classification, it means that the child will not be allowed to go to college.

LabCode(Provided on the original webpage ):
Copy code

% Exercise 4 -- Logistic Regression

Clear all; close all; CLC

X = load ('ex4x. dat ');
Y = load ('ex4y. dat ');

[M, N] = size (X );

% Add intercept term to X
X = [ones (M, 1), X];

% Plot the training data
% Use different markers for positives and negatives
Figure
Pos = find (y); neg = find (y = 0); % find is a found vector, and the result is the number of the value when the bracket value of the find function is true.
Plot (x (Pos, 2), x (Pos, 3), '+ ')
Hold on
Plot (x (neg, 2), x (neg, 3), 'O ')
Hold on
Xlabel ('exam 1 score ')
Ylabel ('exam 2 score ')

% Initialize fitting parameters
Theta = zeros (n + 1, 1 );

% Define the Sigmoid Function
G = inline ('1. 0./(1.0 + exp (-z ))');

% Newton's method
Max_itr = 7;
J = zeros (max_itr, 1 );

For I = 1: max_itr
% Calculate the hypothesis Function
Z = x * Theta;
H = g (z); % converts to a logistic function

% Calculate gradient and Hessian.
% The formulas below are equivalent to the summation Formulas
% Given in the lecture videos.
Grad = (1/m). * x' * (H-y); % vector representation of gradient
H = (1/m). * x' * diag (h) * diag (1-H) * X; % vector representation of Hessian Matrix

% Calculate J (for testing convergence)
J (I) = (1/m) * sum (-y. * log (h)-(1-y). * log (1-H); % vector representation of the loss function

Theta = theta-H \ grad; % is that true?
End
% Display Theta
Theta

% Calculate the probability that a student
% Score 20 on Exam 1 and score 80 on Exam 2
% Will not be admitted
Prob = 1-g ([1, 20, 80] * theta)

% Plot the Sub-interface
% Plot Newton's method result
% Only need 2 points to define a line, so choose two endpoints
Plot_x = [min (x (:, 2)-2, max (x (:, 2) + 2];
% Calculate the demo-boundary line
Plot_y = (-1./theta (3). * (theta (2). * plot_x + theta (1 ));
Plot (plot_x, plot_y)
Legend ('admitted', 'not admitted', 'demo-boundary ')
Hold off

% Plot J
Figure
Plot (0: MAX_ITR-1, J, 'O -- ', 'markerfacecolor', 'R', 'markersize', 8)
Xlabel ('iteration'); ylabel ('J ')
% Display J
J

Copy code

References:

Http://openclassroom.stanford.edu/MainFolder/DocumentPage.php? Course = deeplearning & Doc = exercises/ex4/ex4.html

Author: tornadomeet Source: http://www.cnblogs.com/tornadomeet welcome to repost or share, but please be sure to declareArticleSource.
Classification: Machine Learning