Machine Learning---logistic regression

This chapter mainly explains the principle of logistic regression and its mathematical derivation, the logistic has 3 different forms of expression, and now I will unfold these different forms, and its effect in the classification. And compare these three kinds of forms.

These three forms of loss function are written below:

The following are the three kinds of loss function gradient form:

The first form and the third form are equivalent and are deduced as follows:

Steepest descent

The previous section has described the update formula for the steepest descent method, as follows:

The following will give the code this easy to understand:

Main.m

<span style= "Font-family:times New Roman;" >[D,B] = Load_data ();%%% run exp and log convex logistic regression%%%x0 = Randn (3,1);    % initial Pointalpha = 10^-2;        % Step Lengthx = grad_descent_exp_logistic (d,b,x0,alpha);% Run log convex logistic regressionalpha = 10^-1;        % Step lengthy = grad_descent_log_logistic (D,b,x0,alpha);%%% plot everything, pts and lines%%%plot_all (D ', b,x,y); </ Span>

Load_data (). M

<span style= "Font-family:times New Roman;" > function [A, b] = Load_data ()        data = Load (' Exp_vs_log_data.mat ');        data = Data.data;        A = Data (:, 1:3);        A = a ';        b = Data (:, 4);    End</span>

GRAD_DESCENT_EXP_LOGISTIC.M

<span style= "Font-family:times New Roman;" >function x = grad_descent_exp_logistic (d,b,x0,alpha)        % initializations        x = x0;        iter = 1;        Max_its = n;        Grad = 1;        m=22;  while Norm (grad) > 10^-6 && iter < max_its                        % compute gradient sum=0              ;            For i=1:22                z=b (i) * (D (:, i) ' *x);                Tmp1=exp (-Z);                Tmp2=-b (i) *d (:, i) ';                SUM=SUM+TMP1*TMP2 ';            End            grad= (1/22) *sum;         % your code goes here!            x = X-alpha*grad;                        % Update iteration count            iter = iter + 1;        End    End</span>

GRAD_DESCENT_LOG_LOGISTIC.M

<span style= "Font-family:times New Roman;" >function x = grad_descent_log_logistic (d,b,x0,alpha)        % initializations        x = x0;        iter = 1;        Max_its = n;        Grad = 1;        m=22;  while Norm (grad) > 10^-6 && iter < max_its            sum=0;            For i=1:22                z=b (i) * (D (:, i) ' *x);                Tmp1=exp (-Z)/sigmoid (z);                Tmp2=-b (i) *d (:, i) ';                SUM=SUM+TMP1*TMP2 ';            End            grad= (1/22) *sum;            x = X-alpha*grad;            % Update iteration count            iter = iter + 1;        End    End</span>

PLOT_ALL.M

<span style= "Font-family:times New Roman;" >function Plot_all (a,b,x,y)                % plot points        ind = Find (b = = 1);        Scatter (A (ind,2), A (ind,3), ' LineWidth ', 2, ' Markeredgecolor ', ' B ', ' markerfacecolor ', ' none ');        Hold on        ind = Find (b = =-1);        Scatter (A (ind,2), A (ind,3), ' LineWidth ', 2, ' Markeredgecolor ', ' r ', ' Markerfacecolor ', ' none ');        Hold on                % plot separators        S =[min (A (:, 2)):. 01:max (A (:, 2))];        Plot (S, (-X (1)-X (2) *s)/x (3), ' m ', ' linewidth ', 2);        Hold on                plot (s, (Y (1)-Y (2) *s)/y (3), ' K ', ' linewidth ', 2);        Hold on                set (GCF, ' Color ', ' w ');        Axis ([(Min (A (:, 2))-0.1) (Max (A (:, 2) + 0.1) (Min (A (:, 3))-0.1) (Max (A (:, 3) + 0.1)])        box off                % Graph info LA BELs        xlabel (' a_1 ', ' Fontsize ', +)        ylabel (' a_2  ', ' Fontsize ', ')        set (Get (GCA, ' Ylabel '), ' Rotation ', 0)            end</span>

result Diagram

where the black line is the second loss function, color lines are the first loss function.
Resources----------------Code and datasets See Resources

University of Chinese Academy of Sciences West Lake Campus

Machine Learning---logistic regression