Implement linear regression and logistic regression in MATLAB

This article mainly explains how to implement linear regression and logistic regression in MATLAB, and does not involve formula derivation. The specific formula and derivation are related to machine learning articles and videos. We recommend that you read the Open Course of Andrew Ng.

1. Linear Regression)

Method 1: use the formula:

Function [Theta] = linearreg () % linear regression. X = [1 1; 1 2; 1 3; 1 4]; % note that the first column is 1, that is, X0 = 1, and the second column is x1y = [1.1; 2.2; 2.7; 3.8]; A = inv (x' * X); Theta = A * x' * Y; % according to the formula Theta = (x' * X) ^ (-1) * x' * Y; End

This method is the simplest, but the formula derivation process is complex.

Method 2: Use the Gradient Descent Method for Iteration

 

Function Theta = linearregression () % gradient descent method to find the most appropriate Theta, making J minimum Options = optimset ('gradobj ', 'on', 'maxiter', 100 ); inittheta = [1] '; Theta = fminunc (@ costfunc, inittheta, options); end % function [J, gradient] = costfunc (theta) % J is the cost function. % Y = theta (0) * x0 + theta (1) * x1; find the best Theta to fit the curve. % Makes J the smallest Theta the best thetax = [1; 2; 3; 4]; y = [1.1; 2.2; 2.7]; M = size (X, 1); hypothesis = theta (1) + theta (2) * X; Delta = hypothesis-y; j = sum (delta. ^ 2)/(2 * m); gradient (1) = sum (delta. * 1)/m; % X0 = 1; gradient (2) = sum (delta. * X)/m; End

Both methods use data:

X = [1; 2; 3; 4];
Y = [1.1; 2.2; 2.7; 3.8];

Of course, you can replace it with other data when using it. the result obtained by both methods is

 

theta =    0.3000    0.8600

You can learn linear functions:
Y = 0.3000 + 0.8600 * X;

 

Ii. Logistic Regression)

Method 1: Use the glmfit () function provided by MATLAB ():

 

Function Theta = logisticregression () % logistic regression parameter Theta. You can use the glmfit function of MATLAB to find x = [0.0 0.1 0.7 1.0 1.1 1.3 1.4 1.7 2.1] '; y = [0 0 1 0 0 0 1 1 1] '; Theta = glmfit (x, [y ones ()], 'binomial', 'link ', 'logit ') End

 

Method 2: Use the Gradient Descent Method for Iteration

 

Function Theta = logisticreg () % gradient descent method to find the most appropriate Theta, making the cost function J the smallest Options = optimset ('gradobj ', 'on', 'maxiter', 100 ); inittheta = [0] '; Theta = fminunc (@ costfunc, inittheta, options); end % function [J, gradient] = costfunc (theta) X = [0.0 0.1 0.7 1.0 1.1 1.3 1.4 1.7 2.1] '; y = [0 0 1 0 0 0 1 1 1 1]'; M = size (X, 1); TMP = theta (1) + theta (2) * X; % Theta 'xhypothesis = 1. /(1 + exp (-TMP); % logistic functiondelta = Log (hypothesis + 0.01 ). * Y + (1-y ). * log (1-hypothesis + 0.01); % add 0.01 to Prevent X from being 0j =-sum (DELTA)/m; gradient (1) = sum (hypothesis-y) /m; % X0 = 1; gradient (2) = sum (hypothesis-Y ). * X)/m; % Theta = theta-A * gradient; gradient =-J' (theta) End

Both methods use data:

X = [0.0 0.1 0.7 1.0 1.1 1.3 1.4 1.7 2.1 2.2] ';
Y = [0 0 1 0 0 0 1 1 1 1] ';

Note that values of Y can only be 0 and 1.

Expected result:

 

theta =   -3.4932    2.9402

You can learn the functions:

Y = 1/(1 + exp (3.4932-2.9402 * X ));

 

Implement linear regression and logistic regression in MATLAB