Reprint Deep Learning: Two (linear regression practice)

Objective

This is the practice of multivariate linear regression, which is practiced in the simplest two-dollar linear regression, referring to the Stanford University's teaching network http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course= Deeplearning&doc=exercises/ex2/ex2.html. The subject is given 50 data sample points, of which X is the age of the 50 children, aged 2 to 8 years old, the age can be presented in decimal form. Y for the 50 children of the corresponding height, of course, is also expressed in decimal form. The problem now is to estimate the height of children aged 3.5 and 7, based on these 50 training samples. By drawing out the distribution of the training sample points, it can be intuitively found that this is a typical linear regression problem.

Introduction to MATLAB functions:

Legend

For example, Legend (' Training data ', ' Linear regression '), which represents the meaning of each curve marker in the image, where the first curve of the image (actually a discrete point) represents the training sample data, The second curve (actually a straight line) represents a regression curve.

Hold on, hold off:

Hold on refers to the opening of the drawing paper in the case of the previous picture, allowing the curve to continue to be drawn. Hold off refers to the drawing of the previous painting.

Linspace:

For example Linspace (-3, 3, 100) refers to the 3 to 3 of the 100 number, uniform selection, that is, linear selection.

Logspace:

For example, Logspace (-2, 2, 15), refers to the 10^ (-2) to 10^ (2) between the selection of 15 numbers, these numbers according to the exponential size of the selection, that is, the exponent part is evenly selected, but because all of the 10 is the base of the exponent, so the final is to obey the exponential distribution selected.

Experimental results:

Training sample scatter and regression curve prediction graphs:

The surface graph between the loss function and the parameter:

Contour plot of loss function:

Program code and comments:

The normal equations method is used to solve:

Percent= load (' ex2x.dat '= Load (' ex2y.dat ');p lot (x, y,' * ') xlabel (' height ') Ylabel (' age '= [Ones (size (x), 1), x];w=inv (x ' *x) *x ' *yholdon%plot (x,0.0639*x+0.7502 ) Plot (x (:,2), 0.0639*x (:, 2) +0.7502)% corrected code

Using gradient descend Process solution:

% Exercise 2Linear Regression% Data is roughly based on 2000CDC Growth Figures% forBoysPercent X refers to a boy's age% y is a boy's height in meters%clear all; close all; Clcx= Load (' Ex2x.dat '); y = Load (' Ex2y.dat '); M= Length (y); %Number of training examples%Plot the training datafigure;% Open aNewFigure Windowplot (x, Y,' O '); Ylabel (' Height in meters ') Xlabel (' Age in Years ')%Gradient Descentx= [Ones (M, 1) x]; %Add A column of ones to Xtheta= Zeros (Size (x (1,:))) ';% Initialize fitting PARAMETERSMAX_ITR = 1500; Alpha= 0.07; forNum_iterations = 1: Max_itr%This is a vectorized version of the%Gradient Descent Update formula% It ' s also fine to use the summation formula from the videos%Here is the gradient grad= (1/m). * x ' * ((x * theta)-y); %Here is the actual update theta= Theta-alpha. *grad; % sequential update:the wrong to DoGradient Descent% Grad1 = (1/m). * X (:, 1) ' * ((x * theta)-y); % theta (1) = theta (1) + alpha*Grad1; % Grad2 = (1/m). * X (:, 2) ' * ((x * theta)-y); % theta (2) = Theta (2) + alpha*Grad2;end%Print theta to Screentheta%Plot the linear fithold on;%Keep previous plot visibleplot (X (:,2), X*theta, '-') Legend (' Training data ', ' Linear regression ')%mark the meaning of each curve mark in the image hold off% don ' t overlay any more plots on the this figure, referring to closing the previous picture% Closed form solution forReference% you'll learn about Thismethod in the future Videosexact_theta= (x ' * x) \x ' *y% Predict values forAge 3.5 and 7Predict1= [1, 3.5] *Thetapredict2= [1, 7] *Theta%Calculate J Matrix%Grid over which we'll calculate jtheta0_vals= Linspace (-3, 3, 100); Theta1_vals= Linspace (-1, 1, 100);% initialize j_vals to a matrix of 0 ' sj_vals =Zeros (Length (theta0_vals), Length (theta1_vals)); fori = 1: Length (theta0_vals) forj = 1: Length (theta1_vals) T=[Theta0_vals (i); Theta1_vals (j)]; J_vals (I,J)= (0.5/m). * (x * t-y) ' * (x * t-y);EndEnd%Because of the meshgrids work in the surf command, we need to% transpose j_vals before calling surf, orElseThe axes'll be flippedj_vals= J_vals ';%Surface Plotfigure;surf (theta0_vals, Theta1_vals, j_vals) Xlabel (' \theta_0 '); Ylabel (' \theta_1 ');%Contour plotfigure;% Plot j_vals as contours spaced logarithmically between 0.01 and 100Contour (theta0_vals, Theta1_vals, J_vals, Logspace (-2, 2, 15))%Draw the Contour line Xlabel (' \theta_0 '); Ylabel (' \theta_1 ');% is similar to an escape character, but can only be a parameter 0~9

Resources:

http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course=DeepLearning&doc=exercises/ex2/ Ex2.html

Tornadomeet Source: Http://www.cnblogs.com/tornadomeet Welcome to reprint or share, but be sure to declare the source of the article.

Reprint Deep Learning: Two (linear regression practice)