The foundation of machine learning--covariance matrix

Definition of covariance

For the general distribution, the direct generation of E (X) and the like can be calculated, but really give you a specific numerical distribution, to calculate the covariance matrix, according to the formula to calculate, it is not easy to react. Here's an example of how the covariance matrix is calculated.

Remember, X , Y is a column vector that represents the number of possible occurrences of each sample in each case. For example given

theXrepresentsxthe number of axes that may appear,Yrepresentsythe axis may appear. Note Here is the key, given the4sample, each sample is two-dimensional, so there may only beXand theYtwo different dimensions. So

To describe in Chinese is:

covariance(i,j) =(SectionIall elements of a column-SectionIthe mean value of the column)*(SectionJall elements of a column-SectionJthe mean value of the column)

It's only x, y two columns, so the resulting covariance matrix is 2x2 Matrix, each of the following is calculated as an element:

So, by definition, the covariance matrix for a given 4 two- dimensional sample is:

with matlab Calculate This example

z=[1,2;3,6;4,2;5,2]

CoV (z)

Ans =

2.9167-0.3333

-0.3333 4.0000

matlab 3 times. Therefore, the of the covariance matlab calculation formula

covariance(i,j) =(SectionIcolumn all elements-SectionIcolumn mean value)*(SectionJcolumn all elements-SectionJcolumn mean value)/(number of samples-1)

the following gives a4Dimension3example of a sample, note4dimensional samples and symbolsx, yIt's not going to matter.x, yrepresents a two-dimensional,4dimension is applied directly to the calculation formula, withoutx, yso it's a confusing expression.

(3) with the matlab Calculation Validation

Z=[1 2 3 4;3 4 1 2;2 3 1 4]

CoV (Z)

Ans =

1.0000 1.0000-1.0000-1.0000

1.0000 1.0000-1.0000-1.0000

-1.0000-1.0000 1.3333 0.6667

-1.0000-1.0000 0.6667 1.3333

         !

Reference:

[1] Http://en.wikipedia.org/wiki/Covariance_matrix

[2] http://www.cnblogs.com/cvlabs/archive/2010/05/08/1730319.html

The foundation of machine learning--covariance matrix