Deep learning Learning (b) Matalab operation of linear regression

Continue learning http://www.cnblogs.com/tornadomeet/archive/2013/03/15/2961660.html

The title is: 50 Data sample points, of which X is the age of the 50 children, ages 2 to 8 years old, the age can be presented in decimal form. Y for the 50 children of the corresponding height, of course, is also expressed in decimal form. The problem now is to estimate the height of children aged 3.5 and 7, based on these 50 training samples.

The data can be http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course=DeepLearning&doc=exercises/ex2/ ex2.html Download

The author has given the code:

The normal equations method is used to solve:

1%%Method One2x = Load ('Ex2x.dat');3y = Load ('Ex2y.dat');4Plot (x, Y,'*')5Xlabel ('Height')6Ylabel (' Age')7x = [Ones (Size (x,2),1), x];%Note that the author of this place is wrong, and the author has hit size (x), which is not right, because size (x) will come out of the two dimensions of the vector x8Percent, we just need the first dimension, and we're going to add another column 1 because this place wx+b becomes W ' x so we have a homogeneous linear equation, so we're going to expand the X into a column of 1. 9W=INV (x'*x) *x'*yTen  on APlot (X (:,2),0.0639*x (:,2)+0.7502The 0.7502 here is the first value of the W vector to be obtained, that is, the Wx+b b,w the second value is Wx+b W.

Method Two:

1 clear all, close all, CLC2x = Load ('Ex2x.dat'); y = Load ('Ex2y.dat');3m = Length (y); %Number of training examples4%Plot the training data5Figure % Open aNewFigure Window This figure also can not write, no influence6Plot (x, Y,'o');%use a circle to represent a data point7Ylabel ('Height in meters')%What does it mean to write the Y value?8Xlabel ('Age in years')9 Ten%Gradient Descent Onex = [Ones (M,1) x]; %add a column of ones to x x first adds a column of 1, that is, each data point adds one dimension, and this dimension is 1, A% equal to the required linear equation is the homogeneous w'X=y,x is a two-dimensional, Y represents a training-based W'y value of x prediction -theta = Zeros (Size (x (1,:)))';% Initialize fitting parameters W'Initialize to [0;0] -Max_itr = the; theAlpha =0.07;%Learning Rate -  -  forNum_iterations =1: Max_itr -Grad = (1/m). * x'* ((x * theta)-y);%grd specifically how to calculate can see the following deduction, but here 1/m don't know how to get out, +% mine is 2m, note grad is avector of 1. And the form inside the formula -% is a bit different because in the equation Xi represents a vector, where x is a matrix, and each row represents a sample, so here's The X in front of the code.'followed by x, +% is exactly the opposite in the formula. *is the dot multiplication, not the inner product, the result of the inner product of the vector is the number, this is a vector Atheta = theta-alpha. * GRAD; % here if the method that makes the grad=0 to get the parameter is the method that precedes it, this is not grad=0, instead of iterating over and over again, to find the most value.  at End -Hold on; %Keep previous plot visible -Plot (X (:,2), X*theta,'-')%This is the graph of the regression curve. -Legend'Training Data','Linear regression')%mark the meaning of each curve marker in the image, that is, the meaning of the circle or line segment represented by each data point . -Hold off% Don'T overlay any further plots on the this figure, which means to turn off the front image -% Closed Form Solution forReference in% you'll learn about ThisMethodinchFuture Videos -Exact_theta = (x'* x) \x'* y%I don't know what that means. to% Predict values forAge3.5and7 +Predict1 = [1,3.5] *Theta -Predict2 = [1,7] *Theta the%Grid over which we'll calculate J *Theta0_vals = Linspace (-3,3, -);% generates one from-3 to 3 with a uniform vector of 100 elements $Theta1_vals = Linspace (-1,1, -);Panax Notoginseng% initialize j_vals to a matrix of0's -J_vals =Zeros (Length (theta0_vals), Length (theta1_vals)); the  fori =1: Length (theta0_vals) +        forj =1: Length (theta1_vals) At =[Theta0_vals (i); Theta1_vals (j)];  theJ_vals (i,j) = (0.5/m). * (x * t-y)'* (x * t-y);% when the value of the parameter is from ( -3,1) to (3,1) +% of the rectangle within the uniform sampling value (take the 100*100 parameters), all sample XI corresponds to each parameter -%the error of the regression equation is a value of J_vals (I,J) $ End $ End -J_vals = J_vals'; -%Surface Plot the Figure ; -Surf (theta0_vals, theta1_vals, j_vals)%draw an image of the parameter and the loss function. Pay attention to using this surf to compare the egg ache, surf (x, y, z) is this,Wuyi%x,y is a vector, Z is a matrix, a mesh made of X, Y ( -*100 points) with each point of Z the% to form a graph, but how does it correspond, where the egg hurts is that the second element of your x and the first element of y are formed by the point Not and Z (2，1) value corresponds!!  -% but and Z (1,2) corresponding!! Because the front forms Z (2，1), the second element of x is the first element of Y with a Wu%so j_vals before the switch.  -Xlabel ('\theta_0'); Ylabel ('\theta_1'); About%Contour Plot $ Figure ; -% Plot j_vals as  theContours spaced logarithmically between0.01and - -Contour (theta0_vals, Theta1_vals, J_vals, Logspace (-2,2, the))%draw a contour line -Xlabel ('\theta_0'); Ylabel ('\theta_1');% is similar to escape character, but can only be up to parameter 0~9

The 18th line of formula: that is, the cost function w is the parameter of this equation, the equation for W derivative is the code grad.
The result of the request is

Experimental results: Training sample scatter and regression curve prediction diagram:

The surface graph between the loss function and the parameter:

Deep Learning Learning (b) Matalab operations for linear regression