A reflection on the multivariate linear regression and the analysis of principal components in "foreign articles"

The first two days in the study of the option portfolio, suddenly feel a little new understanding of statistics, so write a little about the linear aspects of the multivariate, to wait for later use.

1. Basic forms of multivariate linear regression:

A dependent variable, such as the temperature in an area, is thought to be related to several other independent variables, such as altitude, sunlight, humidity, and so on. I'm here to interpret these arguments as the corresponding instrument. Assuming there are n observations, the resulting data is:

It can be understood here that in each observation we have the portfolio of Y and the corresponding values of each instrument. Now, we need to find out how many instrument the portfolio are made of. Moreover, because the observations are random, the composition is fixed !

Here, there are two things to consider:

1) n<k (number of independent variables)

For example, assuming n=2, then this problem can be written in order to find the a,b,c in the following equation group

According to the knowledge of linear algebra, we know that the solution of a,b,c is not unique. There are so many instrument that these instrument give different weights to the target result y. (Of course, all of this assumes that the properties of the Matrix are good)

2) n>k

Using the knowledge of linear algebra again, A,b,c is either unique (collinearity), or cannot be solved, what then? We introduce the concept of a residual e.

Then the equations above will become:

In order to get the unique solution of a,b,c, we introduced the idea of least squares, which is to minimize the sum of squares of E!

On the derivative of B (parameter) and the order is 0, then we can get the unique B (i.e.: the only a,b,c solution)

Because this solution is obtained by using the least squares as the target conditions, there are random factor e in the least squares, so the solution of B can only be considered according to the estimate of the existing observations. Therefore, we need to know the range of the parameters of B, which has the standard deviation of the parameters.

Further, we know that these parameters are estimated to obey the t distribution, and the degrees of freedom is n-k. Degrees of freedom can be understood as, if you add n-k more instrument, then you can fully determine the value of B based on the resulting equations, and no least squares are required.

2. Main component Analysis thought:

From the above analysis, we know that we are actually using a given instrument composition to simulate y this portfolio. So, can you use other instrument to replace the original, and then also get y? The answer is yes.

This is a bit of orthogonal decomposition, as in the above example, if there are 3 instrument, then I can find the replacement instrument in three dimensions, and these three new instrument can be a linear combination of the original instrument perfectly. This is actually one of the things that the main component analysis is doing. Note that this new instrument has no linear relationship with each other (different dimensions)!

The orthogonal decomposition is:

It can be proved that the covariance matrix is that they are linearly unrelated!

A reflection on the multivariate linear regression and the analysis of principal components in "foreign articles"