On the compression Perception (ix): Norm and sparsity

Problem:

when finding sparse solutions for compression-aware problems, it is generally used 0 Norm or 1 norm to build a mathematical model. So why is the 0 - norm or 1 - norm able to get sparse solutions?

Interpretation and Analysis: 1, Norm

It is common to have l 0 norm,l1 norm,l2 norm, I often want to be the L0 norm equivalent to the L1 norm to solve, because l The 1 norm solution is a convex optimization problem, and the L0 Norm Solution is a NP-hard problem.

The l0 norm refers to the number of non-0 elements in X, which is the sparsity of x, and if X is sparse, the l0 norm equals K;

The L1 norm refers to the and of the modulus of all elements in X;

The L2 norm refers to the sum of squares of the square of all the elements in X and the sum of squares, this band formula can be, it represents the concept of distance;

There is also the infinity norm, which refers to the maximum value of the element modulus in X.

2. sparsity

The concept of "K sparse" is often mentioned in the compression perception, which is very easy to understand: that is, for a vector of length n (in fact, an n-dimensional discrete-value signal), its n element values are only k nonzero, where k<<n, At this point we call this vector is K sparse or strict k sparse, in practice to do strict k sparse is not easy, in general, as long as the other values except this K value is very small, we think the vector is sparse, then the strict K sparse and called it K sparse bar.

Why should we talk about the problem of sparsity? Because if the signal is sparse, then it is compressible, that is, there are so many 0 inside, I just record those non-0 value and its position is good.

Of course, the reality of the signal itself is generally not sparse, but after a transformation, in a group of base is sparse, which is the sparse representation of the signal.

Sparsity is the premise of compression perception.

3. The relationship between Norm and sparsity

Compression-aware mathematical model:

The geometric representations of mathematical models established by different norms are as follows:

Ax=b is a linear projection process, so in space is a straight line;

and | | x| | P is represented as a geometry in space, and its shape is related to p;

When 0<p<1, the LP ball is convex, and when the radius of the ball increases gradually, the intersection with the line will be on the axis, and such a point is sparse, because the point on the axis, in addition to the axis coordinates, other coordinate values are 0;(in n-dimensional space, the points on the axis are sparse)

When P=1, the LP Ball is a rhombus, and under certain conditions it also leads to a sparse solution;

When P>1, LP, when the expansion of the image and the tangent point must not be located on the axis, that is, the solution is not sparse, such as L2 for the ball.

Reference:

http://blog.csdn.net/jbb0523/article/details/40268943

On the compression Perception (ix): Norm and sparsity