Stanford machine learning lab 1

It is decided that machine learning is under system learning, and Stanford courseware is the main line.

Notes1 is part of the http://www.stanford.edu/class/cs229/notes/cs229-notes1.pdf about Regression

1. Linear Regression

For example, if the House Price is predicted and the data cannot be found on the Internet, use five data points for the experiment.

House.txt, 5 data records, number of bedrooms, Price.

Area bedrooms price
2104 3 400
1600 3 330
2400 3 369
1416 2 232
3000 4 540

 

Use R to display data

> House = read.table('house.txt ', header = T)
> House
Area bedrooms price
1 2104 3 400
2 1600 3 330
3 2400 3 369
4 1416 2 232
5 3000 4 540
> House $ Area
[1] 2104 1600 2400 1416 3000
> Plot (House $ area, house $ price)
>

 

> Fit = LM (House $ price ~ House $ area) // try linear regression price = W * area + B
> Abline (FIT)

> Summary (FIT)

Call:
Lm (formula = house $ price ~ House $ Area)

Residuals:
1 2 3 4 5
25.80 39.02-54.08-28.60 17.85

Coefficients:
Estimate STD. Error T value PR (> | T |)
(Intercept) 26.78988 78.20681 0.343 0.7545
House $ area 0.16512 0.03588 4.602 *
---
Signif. Codes: 0 '***** '000000' ** '000000' * '000000'. '000000' 1

Residual standard error: 45.64 on 3 degrees of freedom
Multiple r-squared: 0.8759, adjusted R-squared: 0.8345
F-statistic: 21.18 on 1 and 3 DF, p-value: 0.01929

 

Therefore, the fitting formula obtained by R is

Price = 26.78988 + 0.16512 * Area

 

If we consider the impact of area and bedrooms on house prices at the same time

Using R Multiple Regression

> Fit = LM (House $ price ~ House $ area + house $ bedrooms)
> Summary (FIT)

Call:
Lm (formula = house $ price ~ House $ area + house $ bedrooms)

Residuals:
1 2 3 4 5
25.80-12.02-24.10 5.16 5.16

Coefficients:
Estimate STD. Error T value PR (> | T |)
(Intercept)-70.43460 59.50462-1.184 0.358
House $ area 0.06384 0.04458 1.432 0.288
House $ bedrooms 103.43605 40.09826 2.580 0.123

Residual standard error: 26.87 on 2 degrees of freedom
Multiple r-squared: 0.9713, adjusted R-squared: 0.9426
F-statistic: 33.87 on 2 and 2 DF, p-value: 0.02868

Price =-70.43 + 0.06384 * area + 103.43605 * bedrooms

This is a big difference from the courseware, mainly because the data set here is too small and there are only five data points.

 

C ++ lab

Considering that the above regression is essentially a least square problem. If we solve the least squares Ax = B from the perspective of linear algebra, here we will use eigen to do the experiment, which corresponds to the above two examples of 1 yuan and multivariate linear regression respectively.

Always 1. for the exam area, if the binary regression corresponds to the price

 /**  * ===================================================== ========================================================  *  * \ File stanford1.cc  *  * \ Author chenghuige  *  * \ Date 2011-02-27 15:27:07. 614842  *  * \ Description: Stanford machine learning experiment  *  Area bedrooms price  2104 3 400  1600 3 330  2400 3 369 1416 2 232  3000 4 540    * ===================================================== ========================================================  */  # Define   Private   Public  # Define   Protected   Public # Include <iostream> # include < String > # Include <vector> # include <fstream> # include <algorithm> # include <boost/progress. HPP> # include <glog/logging. h> # include <gflags/gflags. h> # include "Debug_help.h" # Include"Utils/matrix_help.h"  Using   Namespace STD; define_string (type, "Simple" , "" ); VEC linear_regression ( Const Mat &, Const VEC & B ){ // Ax = B Least squar sort or other method return x      Return A. jacbisvd (computethinu | computethinv). Solve (B );} Void Run () {mat data (5, 4 ); // 5 data points, each with 3 attrib with a const attrib Data <1, 2104, 3,400, 1, 1600, 3,330, 1, 2400, 3,369, 1, 1416, 2,232, 3000, cout < "The experiment data is as follows: \ n" <Data <Endl; cout < "The result of one-dimensional linear regression is as follows, corresponding to the constant coefficient and area coefficient :" <Endl; cout <linear_regression (data. leftcols (2), Data. COL (3) <Endl; // One-dimensional linear regression Cout < "The result of binary linear regression is as follows, corresponding to the constant coefficient and area coefficient and bedrooms coefficient :" <Endl; cout <linear_regression (data. leftcols (3), Data. COL (3) <Endl; // Binary Linear Regression } Int Main ( Int Argc, Char * Argv []) {flags_logtostderr = True ; Google: initgooglelogging (argv [0]); Google: installfailuresignalhandler (); Int S = Google: parsecommandlineflags (& argc, & argv, False ); Boost: progress_timer timer; run (); Return 0 ;}

[Chg @ localhost bin] $./stanford1
The experiment data is as follows:
1 2104 3 400
1 1600 3 330
1 2400 3 369
1 1416 2 232
1 3000 4 540
The linear regression result is as follows, corresponding to the constant coefficient and area coefficient:
26.7899
0.165119
The result of binary linear regression is as follows, which corresponds to the constant coefficient and area coefficient and bedrooms coefficient:
-70.4346
0.0638434
103.436
0.00 s

The result is consistent with that of R.