0 descending from gradient to Fista

What is Fista? A Fast ISTA (iterative shrinkage-thresholding algorithm). That is, a fast iterative threshold shrinkage algorithm.

Consider the following linear conversion problem: B = Ax + W (1)

For example, in the image blur problem, A is a fuzzy template (from the non-blurred image through conversion), B is a fuzzy image, W is noise. Also, A and b are known, X is the coefficient to be asked.

The traditional method of solving this problem is the least squares, the thought is very simple and rough: Make the reconstruction error | | ax-b| | 2 min. That

to f (x) = | | ax-b| | 2 derivation, the derivative can be: F ' (x) = 2AT (ax-b). For this problem, the minimum value (the function f (x) is the convex function and the minimum value) can be obtained by making the derivative zero.

1) If a is a non-singular matrix, i.e. a reversible, then the exact solution of the problem can be x=a-1b.

2) If a is a singular matrix, i.e. a is not reversible, then the problem has no exact solution. Back to the second, we ask for an approximate solution to be good, | | ax-b| | 2<= ?.

Where, | | x| | 1 is a penalty term used to normalize the parameter x. This example uses the L1 norm as a penalty, and it is expected that X is as sparse as possible (as few as 0 elements), i.e. B is a sparse representation of a. | ax-b| | 2<= is the constraint condition, that is, the minimum reconstruction error. The question (3) can also be described as:

0 descending from gradient to Fista