P1 Problem Solving algorithm
1. BP
Using the Linear programming toolbox of matlab, i.e. Linprog
Main Idea: How to convert P1 problem into linear programming problem
That is, from 3.1 to 3.2.
x=[u; v], where u,v are positive, a=u-v
a=[phi,-phi] (can not show the symbol, read it just fine ...) )
B=s
Then b=ax= (u-v) =a=s
Solution: X0=linprog (c,[],[],a,b,zeros (2*p,1));
The optimal solution of the problem P1 is x0 (1:p)-x0 (p+1:2p)
Note: to remove the absolute value here, prove the Following:
2. bpdn-based Tracking De-noising
Using Quadprog
X0=quadprog (b,c,[],[],[],[],zeros (2*m,1));
X0=x0 (1:m)-x0 (m+1:2*m);
As with the BP idea, the P1 problem is converted into two planning problems,
Make X=u-v
The difference is that the BPDN allows the presence of noise and can refer to the differences between Irls-nonoise and Irls.
n=20;m=30; A=randn (n,m); W=sqrt (diag (a ' *a)); for k=1:1:m a (:, k) =a (:, k)/w (k); ends=5;x=zeros (m,1);p os=randperm (m); x (pos (1:s)) =sign ( Randn (s,1)). * (1+rand (s,1)); The sparsity of%x is s, that is, the sparsity is gradually increased to smaxb=a*x;lamda=0.0001*2.^ (0:0.75:15); for i=1:1:21 %{
%BP bb=a ' *b; C=lamda (i) *ones (2*m,1) +[-bb;bb]; B=[a ' *a,-a ' *a;-a ' *a, a ' *a]; X0=quadprog (b,c,[],[],[],[],zeros (2*m,1)); x0=x0 (1:m)-x0 (m+1:2*m); %}
%BPDN bb=a ' *b; C=lamda (i) *ones (2*m,1) +[-bb;bb];% Note the value of lamda, which is taken as 0.05 b=[a ' *a,-a ' *a;-a ' *a, a ' *a]; X0=quadprog (b,c,[],[],[],[],zeros (2*m,1)); X0=x0 (1:m)-x0 (m+1:2*m);
3.IRLS Iterative Weighted least squares
L0 relaxation to l1, get P1, namely BPDN (base tracking noise reduction basis Pursuit denoising), P1 is written as the standard optimization problem is Q1, adding Lagrange multipliers.
General definition Q1 for set lock regression (LASSO)
Main Idea: to facilitate the derivation of the problem Q1,
Enter a, x, b,
The least squares solution of the problem M:
XOUT=INV (2*lambda (ll) *xx+aa) *ab;
Update of the diagonal weighted matrix x, xx is the inverse of X:
Xx=diag (1./(abs (xout) +1e-10));
When | | Stop iteration when less than the preset threshold value
Approximate solution of output problem Q1
n=100; m=200; s=4; sigma=0.1; A=randn (n,m); W=sqrt (diag (a ' *a)); for k=1:1:m, a (:, k) =a (:, k)/w (k); end;x0=zeros (m,1);p os=randperm (m); x0 (pos (1:s)) =sign ( Randn (s,1)). * (1+rand (s,1)); b=a*x0+randn (n,1) *sigma;% the IRLS algorithm for varying lambdaxsol=zeros (m,101); Err=zeros (101,1); Res=zeros (101,1); lambda=0.0001*2.^ (0:0.15:15); Aa=a ' *a; Ab=a ' *b;for ll=1:1:101 xout=zeros (m,1); Xx=eye (m);% Unit diagonal matrix for k=1:1:15 xout=inv (2*lambda (ll) *xx+aa) *ab; Xx=diag (1./(abs (xout) +1e-10)); % disp (norm (xout-x0)/norm (x0)); end; Xsol (:, Ll) =xout; ERR (ll) = (xout-x0) ' * (xout-x0)/(x0 ' *x0);% error Res (ll) = (a*xout-b) ' * (a*xout-b)/sigma^2/n;% residual end;
How to choose Lagrange multiplier?
Rule of thumb: Choose the ratio of the approximate noise deviation value to the standard deviation of the expected non-0 value
Note: optimal when residual and noise power are equal
determine the optimal method: determine the approximate value according to the rule of thumb lambda=0.0001*2.^ (0:0.15:15);
For the calculation, the minimum error corresponds to the optimal
Posl=find (err==min (Err), 1);
Note: BPDN performs slower than Irls
4. LARS (least angle regression stagewise) minimum angular regression distribution
This is going to be a little more.
The thought of forward selection and OMP is close
Forward stagewise feeling is smoother because the step size is smaller
The step size taken by the Lar is between the two, that is, the minimum angle Regression.
The thought of combining both is Lars.
Main Ideas:
is the sparse (regression) coefficient column vector, which is the estimated vector
First take, that is
c is proportional to the correlation between the X and the current residual vector (strange: )
Take |cj| the biggest direction
Algorithm Steps:
- j, and constantly update a=[a,j];
For ease of understanding, you can introduce XA
- Calculate gamma
15.16 Two-style:
3. update mu
4. Seeking Bita
n=100; m=200; s=4; sigma=0.1; A=randn (n,m); W=sqrt (diag (a ' *a)); for k=1:1:m, a (:, k) =a (:, k)/w (k); end;x0=zeros (m,1);p os=randperm (m); x0 (pos (1:s)) =sign (randn (s,1 ). * (1+rand (s,1)); b=a*x0+randn (n,1) *sigma; Err=zeros (n,1); Res=zeros (n,1); Xsollars=lars (a, b);% seek sparse vector for k=1:1:n xout=xsollars (k,:) '; Errlars (k) = (xout-x0) ' * (xout-x0)/(x0 ' *x0), end;function beta = Lars (x, y)% for sparse vectors [n,m]=size (×); nvars=min (n-1,m);??? Mu=zeros (n,1); beta = zeros (2*nvars, m); i=1:m; A=[];k=0;vars=0;gram=x ' *x;while vars<nvars k=k+1; C=x ' * (y-mu); [c,j]=max (abs (C (i))),%c (i) that does not contain the previously taken J J=i (j); A=[a j]; vars=vars+1; I (i==j) =[];% here is not zero, but directly removes J s=sign (c (A)); G=s ' *inv (gram (a,a)) *s; aa=g^ ( -1/2); WA=AA*INV (gram (a,a)) *s; U=x (:, A) *wa; If Vars==nvars gamma=c/aa; else A=x ' *u; Temp=[(c-c (i))./(aa-a (i));(c+c (i))./(aa+a (i))];% minus the j-column vectors in C and a that do not belong to a, resulting in gamma gamma=min ([temp (temp>0); c/aa]); End mu=mu+gamma*u; Beta (k+1,a) = Beta (k,a) + gamma*wa '; endif size (beta,1> k+1 Beta (k+2:end,:) = [];% only takes nvars, beta is sparse vector endend
Expansion: hard threshold, Soft threshold value
If A is the Emirates matrix, then AA (t) =i, So the P0 problem can be written as:
Make
The solution with the smallest error, that is, the closest sparse vector, simply select the first few items by descending the absolute value.
where T () is a hard threshold Value. How to take it?
With the thought of Lars
Lambda is a soft threshold value
P1 Problem Solving algorithm