Python linear regression analysis, python linear practice
I. Linear Regression Theory
1) Basic concepts of Linear Regression
Linear regression is a supervised learning algorithm. It describes the linear correlation between the independent variables and the dependent variables. It can be divided into one-dimensional linear regression and multiple linear regression. Linear regression is a regression between an independent variable and a dependent variable. It can be seen as a special case of linear regression. Linear regression can be used for prediction and classification. From the regression equation, we can see the relationship between independent variables and dependent variables.
The linear regression model is as follows:
The linear regression model is assumed as follows:
(1) The mean value of the error item is 0, and the error item has no linear relationship with the interpretation variable.
(2) The error items are independent and distributed, that is, each error item is independent of each other and the variance of each error item is equal.
(3) Interpreting linear independence between variables
(4) normality hypothesis, that is, the error items are subject to normal distribution.
The above assumptions are the basic conditions for establishing a regression model. Therefore, we need to verify the regression results one by one. If the assumptions are not met, we need to make relevant corrections.
2) solving model parameters
(1) Moment Estimation
Generally, the population parameters are estimated by the sample moment. It is usually the first-order original moment of the sample to estimate the mean of the population, and the second-order center moment to estimate the population variance.
(2) Least Squares estimation
Generally, the Least Square estimation is applicable to variables whose dependent variables are continuous. The most common is Ordinary Least Square, OLS ), the principle is that the selected regression model should minimize the sum of the residual squares of all observed values. The predicted value is represented by the actual value, the sum of the squares of the residual values, and the least square estimation is the value of the obtained parameter to minimize the value of L. The parameter values obtained by linear regression are unique.
(3) Maximum Likelihood Estimation
Maximum Likelihood Estimation is based on probability. It requires that the probability distribution of samples be known. The parameter estimation value maximizes the probability of a large number of samples, which is measured by the likelihood function, the likelihood function is the product of the density functions of each sample. To facilitate the solving of the logarithm, add a negative number to solve the minimum value and obtain the parameter estimation result.
3) advantages and disadvantages of the Model
Advantage: the results are easy to understand and the computing is not complex.
Disadvantage: poor fitting of non-linear data
Ii. Small Example of linear regression using Python
Data comes from web crawlers. The price of commercial housing in Wuhan is the dependent variable and the search volume of Baidu index with several related keywords is the independent variable.
Since there are 98 independent variables in this article, first select the independent variables. First, filter out irrelevant variables through the correlation coefficient matrix, and select variables based on the Pearson correlation coefficient matrix, generally, variables with a correlation coefficient value greater than 0.3 are selected for regression analysis. Because there are many variables in this article, manually filter the variables and then select them using the correlation coefficient, this article selects a variable with a correlation coefficient greater than 0.55 for regression analysis.
After analyzing the correlation coefficient, select 8 variables for the next analysis. The Python code for analysis is as follows:
#-*-Coding: UTF-8-*-#### Required Packagesimport sysreload (sys) sys. setdefaultencoding ('utf-8') import matplotlib. pyplot as pltimport numpy as npimport pandas pdimport statsmodels. api as smdata = pd.read_csv('Hdata.csv ') print datadataset = np. array (data) ###### correlation analysis X = dataset [:, 1: 98] y = dataset [:, 0] cor = np. required coef (dataset, rowvar = 0 )[:, 0] ###### print cor ####### read data after filtering data1 = pd.read_csv('H1data.csv ') dataset1 = np. array (data) ###### filtered variables ###### X1 = dataset1 [:,] Y1 = dataset1 [:, 0] est = sm. OLS (Y1, X1 ). fit () print est. summary ()
The linear regression result is as follows:
OLS RegressionResults =======================================================================Dep. Variable: y R-squared: 0.978Model: OLS Adj. R-squared: 0.974Method: Least Squares F-statistic: 287.5Date: Sat, 08 Apr 2017 Prob (F-statistic): 9.35e-36Time: 15:15:14 Log-Likelihood: -442.82No. Observations: 53 AIC: 899.6Df Residuals: 46 BIC: 913.4Df Model: 7 Covariance Type: nonrobust ======================================================================= coef std err t P>|t| [95.0% Conf. Int.]-----------------------------------------------------------------------x1 -0.3691 0.494 -0.747 0.0459 -1.364 0.626x2 0.3249 0.353 0.920 0.0362 -0.386 1.036x3 1.0987 0.837 1.312 0.0196 -0.587 2.784x4 0.7613 0.790 0.964 0.0340 -0.829 2.351x5 -1.5766 1.099 -1.435 0.0158 -3.789 0.636x6 -0.1572 1.077 -0.146 0.0885 -2.325 2.011x7 3.2003 1.603 1.997 0.052 -0.026 6.427=======================================================================Omnibus: 0.413 Durbin-Watson: 1.748Prob(Omnibus): 0.814 Jarque-Bera (JB): 0.100Skew: 0.097 Prob(JB): 0.951Kurtosis: 3.089 Cond. No. 95.5=======================================================================
From the results of regression analysis, we can see that the model's goodness of fit R-squared = 0.978 indicates that the model has a good fitting effect, and the fit is determined based on its size. The significance of the overall model can be seen from the F statistic. The P value corresponding to the result shows that the F statistic is significantly less than 0.05 (0.05 is the significance level, or 0.01 can be selected ), it indicates that the overall model is significant. Its significance indicates whether the interpreted variables can be interpreted by these interpreted variables. The F test is an overall test, the pass of the F test does not mean that every interpretation variable is significant. The significance of each variable depends on the value of the t-test statistic. The P value corresponding to the t-test statistic is less than 0.05 (0.01 or 0.1 is also acceptable. For details, 0.05 is usually selected for the case analysis, from the results, we can see that the p value of the X6 and X7 variables is greater than 0.05, that is, the effect of these two variables on the interpreted variables is not significant. However, if you only care about the issue of prediction, you can skip it. However, if you have studied the extent to which variables affect interpreted variables, You need to further study them. Next, let's look at the DW value. The DW value is 1.748, indicating that the model does not have auto-correlation. Looking at the JB test statistic value, the JB test statistic tests the normality hypothesis. The P value corresponding to the JB value is 0.951, which is significantly greater than 0.05, it can be considered that the model meets the assumption of normality. This article will not explain the actual meaning of parameters.
The basis for determining the correlation between DW values is as follows:
When DW = 0, the residual sequence has completely positive correlation,
When DW = (), the residual sequence has positive auto correlation,
When DW = 2, the residual sequence has no auto-correlation,
When DW = (2, 4), the residual sequence has a negative auto correlation,
When DW = 4, the residual sequence has completely negative auto-correlation.
The general steps for creating a model are described as follows:
(1) select an appropriate model based on the data representation
(2) parameter estimation method applicable to Model Selection
(3) Check the results of parameters
(4) Explain the results