ICA Study Notes

1. Origin of independent component analysis (ICA:

Independent Component Analysis was first used for blind source signal separation (blind source separation, BBS ). Originated from the "cocktail party problem", it is described as follows: At a noisy cocktail party, many people talk at the same time, there may be background music, but the ears can hear each other's words accurately and clearly. This kind of phenomenon that you can choose the sound you are interested in from the mixed sound and ignore other sounds is called the "cocktail party effect ". 2. Definition of ICA :(ICA uncertainty-without any prior knowledge, W and s cannot be determined at the same time. )The purpose of ICA is to perform a linear decomposition of the observed data into independent statistical components. That is, the basic source signal is restored from the linear hybrid signal. The following is an ICA model: it shows how the observed signal X is formed by the mixing of independent signal s through the hybrid matrix. In actual life, we only know the observed signal X, and the hybrid matrix A and independent signal S are unknown. So what ICA needs to do is to estimate the mixed matrix A and independent source signal s under the conditions that the observed signal is known and the assumption that the number of signals is as small as possible. In this way, we can obtain the Independent Component s from the following formula (2), where W is the inverse matrix of: 3. assumptions:1) component is statistically independent; 2) independent component is a non-Gaussian distribution (the independence of Gaussian distribution is equivalent to the absence of correlation, with at most one Gaussian distribution, I .e. random noise); 3) unknown hybrid matrix A is a square matrix; 4) It is generally assumed that the number of signals observed is not smaller than the number of source signals. (Note: it is best to ignore the noise of each sensor. If the noise is large, the noise source can be considered as an independent source for analysis, which makes the algorithm stronger .) 4. ICA estimation method:(That is, the description of the condition hypothesis: objective function, based on which unsupervised learning is performed) 1) Non-Gaussian Maximization (negative entropy, high-order accumulation-common fourth-order accumulation); 2) Mutual Information minimization; 3) maximum likelihood estimation; 4) Kl divergence; after determining the target function, some algorithms (various adaptive Optimization Algorithms) are used for optimization. 5. application:Non-natural signals are separated in the Meg, hidden factors are found in financial data, noise reduction in natural images, face recognition, image separation, voice signal processing, and remote communication. 6. Several important concepts:1) Independence: two random variables Y1 and Y2 are defined. If they are independent of each other, their combined probability distribution P (Y1, Y2) = P (Y1) P (Y2 ); accordingly, we can introduce that the expectation of the independent variable is also independent, that is, 2) correlation: the covariance of the random variable X and Y is defined as follows: if, it is called X and Y linear independence, that is, irrelevant. Unrelated and independent:The correlation coefficient reflects the linear correlation between two random variables. The non-correlation is the linear relationship. IndependentIt refers to general relationships, so unrelated independence is worse than independence.     7. Two uncertainties of ICA1) Amplitude uncertainty:The magnitude parameter cannot be determined by the recovery signal y (t) = cs (t); 2) Uncertain separation signal Arrangement: The s corresponding to y cannot be restored. 8. Detailed steps for ICA processing are as follows:1) zero mean: 2) White: 3) ICA: the essence of ICA: Make full use of the higher-order statistics of data.