Matlab gradient descent and normal equation to realize linear regression of multivariable

first, related concepts1. Gradient Descent

Since z= x*theta-y is a column vector, Z ' *z is the sum of squares and even plus, which is 2 norm, and if z is a matrix, then the diagonal of Z ' *z is the 2 norm of each column of the z-matrix.

2. Normal equation ( normal equation)

θ= (XTX) -1xty.

Second, the code implementation

2104,3,3999001600,3,3299002400,3,3690001416,2,2320003000,4,5399001985,4,2999001534,3,3149001427,3,1989991380,3,2120001494 , 3,2425001940,4,2399992000,3,3470001890,3,3299994478,5,6999001268,3,2599002300,4,4499001320,2,2999001236,3,1999002609,4,4 999983031,4,5990001767,3,2529001888,2,2550001604,3,2429001962,4,2599003890,3,5739001100,3,2499001458,3,4645002526,3,46900 02200,3,4750002637,3,2999001839,2,3499001000,1,1699002040,4,3149003137,3,5799001811,4,2859001437,3,2499001239,3,229900213 2,4,3450004215,4,5490002162,4,2870001664,2,3685002238,3,3299002567,4,3140001200,3,299000852,2,1799001852,4,2999001203,3,2 39500

......................

Percent Clear and Close figuresclear; Close all; Clcdata = Load (' ex1data2.txt '); x = Data (:, 1:2); y = data (:, 3); m = length (y);% features and set them to zero mean% not y normalization [X mu sigma] = featurenorm  Alize (X);% Add intercept term to x percent in the form of a matrix (theta0) theta ' *xx = [Ones (M, 1) x];%% ================ part 2:gradient descent ================alpha = 0.01;num_iters = 400;theta = Zeros (3, 1); [Theta, J_history] = Gradientdescentmulti (X, y, theta, alpha, num_iters);%convergence: Convergence% Plot the convergence Graphfigu The number of elements in the Re;%numel matrix plot (1:numel (j_history), J_history, '-B ', ' linewidth ', 2); Xlabel (' Numbers of iterations '); Ylabel (' Cost J ');% Display gradient descent ' s resultfprintf (' Theta computed from gradient descent: \ n '); fprintf ('%f \ n ', Theta); f printf (' \ n ');% Estimate The price of a 1650 sq-ft, 3 BR house% ====================== YOUR CODE here ===================== =% Recall The first column of X is All-ones. Thus, it does% not need to be normalized.price = 0; % should change thisxp = [1 1650 3];  XP = [XP (1), (XP (2)-mu (1))/sigma (1), (XP (3)-mu (2))/sigma (2)];p rice = xp*theta;fprintf ([' predicted price of a 1650 sq-ft,         3 BR House ' ... ' (using gradient descent): \ n $%f\n '], price); percent ================ part 3:normal equations ================data = Csvread (' Ex1data2.txt '); X = Data (:, 1:2); y = data (:, 3); m = length (y);% Add intercept term to XX = [Ones (M, 1) x];% Calculate the parameters from The normal Equationtheta = normaleqn (X, y);% Display normal equation ' s resultfprintf (' Theta computed from the normal equat  Ions: \ n '); fprintf ('%f \ n ', theta); fprintf (' \ n ');% Estimate The price of a 1650 sq-ft, 3 BR house% ====================== YOUR CODE here ======================price = 0;  % should change thisxexample = [1 1650 3];p rice = xexample*theta;fprintf ([' predicted price of a 1650 sq-ft, 3 BR House         ‘ ... ' (using normal equations): \ n $%f\n '], price);

...............

function [theta] = normaleqn (X, y)%normaleqn computes the Closed-form solution to linear regression%   normaleqn (x, y) c Omputes the Closed-form solution to linear%   regression using the normal Equations.theta = zeros (Size (X, 2), 1);% 1. for Square A, if it is a non-singular square, there is a inverse matrix inv (A)% 2. For singular matrices or non-matrices, there is no inverse matrix, but you can use PINV (A) for its pseudo-inverse% many times you do not need to ask for inverse matrices, such as% Inv (a) *b% can actually be written as% a\b% B*INV )% can actually be written as% b/a% so that the accuracy after inversion is higher theta = PINV ((X ' *x)) *x ' *y;end

..............

function [Theta, j_history] = Gradientdescentmulti (X, y, theta, alpha, num_iters)% Initialize some useful valuesm = length (y); % Number of training examplesj_history = Zeros (num_iters, 1); for iter = 1:num_iters    %theta is 3*1 is   the first column of column vector x is 1 exactly multiplied by THETA0    % (1*3 X  3*47  -  1*47) ' *    theta = Theta-alpha * ((Theta ' *x '-y ') *x) '/m;      % Save The cost J in every iteration    j_history (iter) = Computecostmulti (X, y, theta); endend

..........

function J = Computecostmulti (X, y, theta)% Initialize some useful valuesm = Length (y); % Number of training EXAMPLESJ = 0; J = SUM (((Theta ' * (X ')-y ') (^2))/(2*m); end

iii. Results1. Cost function

2. Forecast Results

Theta computed from gradient descent:  334302.063993  100087.116006  3673.548451 predicted price of a 1650 sq-ft, 3 BR house (using gradient descent): $289314.620338theta computed from the normal equations:  89597.909545  1 39.210674  -8738.019113 predicted price of a 1650 sq-ft, 3 BR house (using normal equations): $293081.464335

Matlab gradient descent and normal equation to realize linear regression of multivariable