A whole bunch of image processing and machine learning concepts

Population sparsity (population sparse) vs Lifetime sparsity (sparse)

F nxm =w nxD ?x dxm

• d Span style= "Display:inline-block; width:0px; Height:2.563em; " > : Represents the original feature space
• n : Transformed feature space
• m : Number of samples
• f Span style= "Display:inline-block; width:0px; Height:2.563em; " > : Feature matrix (Feature matrix)

Each sample is described with only a few active (not 0) features. Specifically, for each column of the feature matrix (a sample) F(i) , there are very few non-0 elements. All the others are 0. For example, an image can be described by some of the goals contained in it, if there are many possible targets, then at some point, there may only be some. We call it population sparsity (sparse population).

good features should be differentiated so that samples can be differentiated. For example, if you need to differentiate between faces and hands, it is clear that skin color is not a distinguishing feature because both the face and the hands have skin color. But if there is no eye, then it is easy to distinguish between human face or hands, so the eye is a distinguishing feature. So to distinguish the sample, we should choose the unique sample , not everyone has the characteristics . The expression of a slightly academic point is that each feature is only allowed to be activated within a small number of samples . That is, in a feature matrix, each row (a feature) should have only a small number of non-0 elements . The attributes of this feature are called Lifetime sparsity (there is sparse).

High dispersal (highly dispersible)

The distribution of the characteristics of each row (the different values of a feature at different times) should be similar to the distribution of the characteristics of other rows, or each feature should have similar statistical characteristics . Specifically, for each row of the matrix, we take the mean value of the square of all the elements of the line (a different value of a feature at different times of the sample) as a description of its statistical characteristics. Each row has a mean value, so the mean value of each row should be the same, so that all features can be considered to have similar distributions . This attribute we call high dispersal (highly dispersive). But for a good feature description, this attribute is not necessary. However, it can prevent the degradation of features , that is, to prevent the extraction of the same characteristics (if extracted to the same characteristics, then the feature is redundant, and does not increase the amount of information, it is generally required to extract the characteristics are orthogonal). For an over-complete feature expression. High dispersal can be understood as a characteristic that only a few inactive are inactive. For example, PCA encoding generally does not satisfy high dispersal, because large eigenvalues correspond to eigenvectors (that is, feature code) that are mostly active.

A whole bunch of image processing and machine learning concepts