Detailed description of Covariance Matrix

Source: Internet
Author: User

In human face recognition, we often deal with covariance matrices, but we still only know their forms, but their meaning is vague, now I will give a detailed derivation of the covariance matrix based on the covariance of a single variable and different forms under different application backgrounds.

Variable description:

Set to a group of random variables that constitute a random vector.
, Each random variable has m samples, then there is a sample Matrix

(1)

Where
Corresponds to the sample vector of each random vector X,
It corresponds to a vector consisting of all the sample values of the random single variable I.

Covariance between single random variables:

Random Variable
The covariance between them can be expressed

(2)

According to the known sample value, the estimated covariance value is as follows:

(3)

It can be further simplified:

(4)

Covariance Matrix:


(5)

Where
To obtain the covariance matrix expression.

If the mean of all samples is a zero vector, formula (5) can be achieved:

(6)

Note:

1. Each element in the covariance matrix represents the covariance between different components of the random vector X, rather than between different samples. For example, the element CIJ is the reflected random variable XI,
Covariance of XJ.

2. covariance is the second-order statistical feature between variables. If the correlation between different components of a random vector is small, the resulting covariance matrix is almost a diagonal matrix. For some special applications, in order to make the length of the random vector smaller, we can use the principal component analysis method to make the covariance matrix of the transformed variable completely a diagonal matrix, then we can discard some smaller components of energy (the elements on the diagonal lines reflect the variance, that is, the exchange energy ). Especially in the pattern recognition field, when the dimension of pattern vectors is too high, it will affect the generalization performance of the recognition system. This process is often needed.

3. Note that the formula (5) and formula (6) are obtained here) only an estimation of the real value of the random vector covariance matrix is provided (represented by the value of the tested sample, which changes with the value of the sample ), therefore, the covariance matrix is dependent on the sample. The larger the number of samples, the wider the coverage of the sample in the population, the more reliable the covariance matrix is.

4. Just like the relationship between covariance and correlation coefficient, we sometimes introduce a matrix of correlation coefficients to better intuitively know the correlations between different components of a random vector.

 

From: http://blog.csdn.net/faceRec/article/details/1697362

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.