Introduction to sparse representations (up)

Statement

1. Although before listening to the compression perception and sparse expression, in fact, yesterday only formally started to understand, purely novice, if there are errors, please point out the common progress.

2. The main learning materials are Coursera open classes at Duke University--image and video processing, by Pro.guillermo Sapiro the 9th lesson.

3. Because of the understanding of image processing also comes from the course, no serious children have seen a few images of books, some of the terminology can only be expressed in English in the video, forgive me ha!

1. Denoising and MAP

The story starts with Denoising, saying that there is a noise-containing picture on hand Lena, how to remove the noise to get a good clean image?

For the above problem, using the X value to represent the grayscale value of a pixel, we can create such a minimized mathematical model:

Where y is the known observed value, which is the original image that contains the noise, and x represents the unknown value to revert to clean.

The intuitive effect of the first item of the model is that the predictive value x is not too far from the observed value Y. The mathematical explanation is that the value probability of x can be regarded as the Gaussian distribution with Y as mean, that is, the image has Gaussian noise, and the second item is a regular term. The origin is as follows: Assuming that X is a distribution with some prior probability, and now the observed value Y, according to the Bayesian principle, now the x distribution (posterior) is proportional to the product of the prior probability distribution and the Gaussian distribution. If the prior probability distribution is the exponential distribution, and the product takes negative logarithm, we can get the MAP model which is very common in machine learning.

The question now is: what is the best priori (prior)? What form should G (x) take? What is the best space to define an image signal?

In academia, the work has been done so much that the process of exploring this problem can be likened to the evolutionary process of apes into humans:

In the first picture, prior assumes that clean image energy is as small as possible, and X is as small as it can be. In the second picture, prior that the restored image would be smooth, resulting in a combination of Laplacian and low energy, which evolved a step forward. The third picture, prior think to consider edges is not smooth drip, need different situation different treatment ... Sparse and redundant is the problem being discussed, is currently the latest evolutionary version, and then there are some algorithms, although also successfully evolved into human, unfortunately too fat, mobility inconvenience--computationally expensive and difficult. What is the priori of Sparse modeling? To answer this question, you need to understand some basic concepts.

2. sparsity and Lp Norm

1. How to represent sparsity

   Indicates the sparsity of a vector can be norm with LP, for an alpha vector of an element x, LP norm formula and function image as follows:

   We hope that regardless of the size of X, its non-zero penalty is the same , L0 norm just meet this requirement, it means to count the number of non -zero alpha vectors .

2. Sparse Modeling of Signal

   An 8x8 image, which can be represented as a 64-dimensional vector x, how to make a sparse representation? Assume that N = 64:

   The left matrix, D, is a dictionary matrix consisting of K-N-dimensional column vectors. According to the relationship between K and N, it can also be divided into:

   1. K > N:over-complete, which is most common in sparse representations

   2. K = N:complete, such as the Fourier transform and the DCT transform are the case

   3. K < N:under-complete

The intermediate column Vector alpha is a sparse vector characterized by a few 0 items, with only three non-0 items representing the linear combination of the D-matrices corresponding to the line vectors.

The last x vector represents the restored vector.

Atoms represents the column vector for D

In fact, the DCT transform can also be regarded as a sparse representation, its D vector is composed of a fixed and exactly perfect orthogonal base vector, and the alpha vector is also somewhat sparse.

For, assuming that the D matrix > is K N and is full-rank, there must be Ax = B for any N-dimensional vector B (x in the figure). Now, with the constraints of LP norm, restricting only a small number of a's column vectors (atoms as a base, vector b is fixed within a span and becomes an Lp optimization problem:

Using purple to represent a plane, norm a spherical (contour) of the same value with cyan, the problem is as follows: Selecting the norm smallest optimal solution in planar Ax = b plane

When P >= 1 o'clock, there are multiple intersections of the norm ball and the plane of peace. This is a convex optimization problem, which can be solved by Lagrange multiplier.

When 0 < P < 1 o'clock, the feasible solution of norm Ball is very sparse, it is a non-convex optimization problem, it is difficult to solve this kind of problem, but it has very good sparsity.

When p = 0 o'clock, the points on the norm Ball are infinitely contracted in addition to the axes, and the intersection of the planes with the plane is on one axis, and the non-0 coefficient is only one.

Back in the first section of the MAP model, the Sparse Modeling model is that the non-0 coefficients are limited to l (meaning that the solution is within a span of up to l atoms), as close as possible to the plane:

In this way, we use a small number of atoms group to synthesize the real signal, and noise cannot be fitted very well, in the projection to the low-dimensional space of the process played a role in reducing noise.

3. Some issues:

Can the model be changed into L0 norm form and other forms to calculate or approximate?

Is the solution set Alpha vector unique? Can we ask for its approximation? If so, how can we estimate the approximate degree?

What kind of dictionary matrix D should be used in order to better eliminate noise? How is dictionary D determined?

Resources:

[1]:image and video processing, by Pro.guillermo Sapiro 9th lesson

[2] http://hi.baidu.com/chb_seaok/item/bdc0903472229990b80c030f

Introduction to sparse representations (up)