Python tests multivariate collinearity-vif variance enlargement factor

Introduction to Python/crawler/data statistics/Data analysis AI/machine learning/Natural language video tutorial URL https://pythoner.taobao.com/

In a regression equation, if two or more two explanatory variables are highly correlated with each other, the results of the regression analysis may not distinguish between each variable and the dependent variable. For example, we want to know the effect of drug addiction on SAT scores, and we ask whether the object absorbs cocaine or heroin and calculates the coefficients between them using software.

Although the regression coefficients of heroin and cocaine were found, the correlation between them overlapped, making r squared larger, still unable to uncover the real situation.
Because heroin addicts often smoke cocaine, a single drug addict is scarce.

When two variables are highly correlated, we usually use only one of them in the regression equation, or create a new synthetic variable, such as cocaine or heroin.

For example, when a researcher wants to control the students ' overall economic background, they will incorporate both parents ' educational attainment into the equation.

A separate consideration of the education level of the father or mother would cause confusion, and the analysis would be blurred, since the educational attainment of husbands and wives was very much correlated.


multiple collinearity poses problems:
(1) The independent variable is not significant
(2) The positive and negative sign of the parameter estimate has effect
 

co-linear statistics:
(1) Tolerance of tolerance
tolerance<0.1 indicates that there is a serious multiplicity of collinear
(2) variance enlargement factor variance inflation factor (VIF)
VIF>10 indicates that there is a severe multiplicity of collinearity


http://blog.csdn.net/baimafujinji/article/details/49799409

Regression analysis is the most basic method in data mining, in which multivariate linear regression based on ordinary least squares can not have multiple collinearity in the model, otherwise the credibility of the model will be greatly compromised. But technically, how do you determine if there are multiple collinearity between the various features in the model?

First look at a set of data

Can we use this set of data to build multivariate linear regression models? Now the problem with this set of data is not very obvious, we might as well use the simplest cross-scatter plot to look at the data may exist problems. So we plotted in R:

From the scatter plot we can see that each explanatory variable has a more pronounced linear relationship with the interpreted variable, and this is what we want to see. But there seems to be some kind of linear relationship between the two explanatory variables. In fact, if the correlation coefficients between the two explanatory variables are calculated, their linear correlation coefficients are high, which implies a serious multiplicity of collinearity. When constructing multivariate linear regression models, it is very easy to generate multiple collinearity between two explanatory variables as the number of explanatory variables increases. At this point you need to consider whether to remove one of the variables from the model, or even reconsider building the model.

But this method of visual observation can only be used as a precursor to qualitative research, and it is best to have a quantitative approach to help us make the final decision. Look at a set of data for this

Can we use this set of data to build multivariate linear regression models? Again, to plot the scatter plot as follows, the argument seems to be a bit collinearity, but not as obvious as the example above, what to do now?

So we're suggesting a more quantifiable approach to describing the seriousness of the problem, rather than just making perceptual decisions by looking at it visually. Below I demonstrate the method of testing the multiple collinearity in SPSS 22. Import the data first, as shown below

Then click Analyze, Regression, linear ... on the menu bar to enter the linear regression dialog box as shown. When the dependent variable and argument are selected, select "Statistics ..." in the upper-right corner, and then choose "Collinearity Diagnostics" in the new dialog box that pops up

After returning to the dialog on the left, select OK, SPSS gives the results of the linear regression analysis. Let's look at the part of the CCP's linear diagnosis, as shown below, if there is a conditional exponential >10, it indicates collinearity. Now the biggest is 9.659, still in an acceptable range.

If the data we are using is the first to give an example of the volume of freight and industrial and agricultural output, then how to look at it. The red box is marked with a condition index of up to 23.973 (>10), and the visible collinearity is absolutely conclusive!

Python tests multivariate collinearity-vif variance enlargement factor