In human face recognition, we often deal with covariance matrices, but we still only know their forms, but their meaning is vague, now I will give a detailed derivation of the covariance matrix based on the covariance of a single variable and different forms under different application backgrounds.
Variable description:
Set to a group of random variables that constitute a random vector.
, Each random variable has m samples, then there is a sample Matrix
(1)
Where
Corresponds to the sample vector of each random vector X,
It corresponds to a vector consisting of all the sample values of the random single variable I.
Covariance between single random variables:
Random Variable
The covariance between them can be expressed
(2)
According to the known sample value, the estimated covariance value is as follows:
(3)
It can be further simplified:
(4)
Covariance Matrix:
(5)
Where
To obtain the covariance matrix expression.
If the mean of all samples is a zero vector, formula (5) can be achieved:
(6)
Note:
1. Each element in the covariance matrix represents the covariance between different components of the random vector X, rather than between different samples. For example, the element CIJ is the reflected random variable XI,
Covariance of XJ.
2. covariance is the second-order statistical feature between variables. If the correlation between different components of a random vector is small, the resulting covariance matrix is almost a diagonal matrix. For some special applications, in order to make the length of the random vector smaller, we can use the principal component analysis method to make the covariance matrix of the transformed variable completely a diagonal matrix, then we can discard some smaller components of energy (the elements on the diagonal lines reflect the variance, that is, the exchange energy ). Especially in the pattern recognition field, when the dimension of pattern vectors is too high, it will affect the generalization performance of the recognition system. This process is often needed.
3. Note that the formula (5) and formula (6) are obtained here) only an estimation of the real value of the random vector covariance matrix is provided (represented by the value of the tested sample, which changes with the value of the sample ), therefore, the covariance matrix is dependent on the sample. The larger the number of samples, the wider the coverage of the sample in the population, the more reliable the covariance matrix is.
4. Just like the relationship between covariance and correlation coefficient, we sometimes introduce a matrix of correlation coefficients to better intuitively know the correlations between different components of a random vector.
From: http://blog.csdn.net/faceRec/article/details/1697362