Talking about the compression perception (19): MP, Omp and Schmidt orthogonality

Regarding MP, OMP correlation algorithm and the convergence proof, may refer to: http://www.cnblogs.com/AndyJee/p/5047174.html, here simply states the algorithm flow and the difference between them.

Main content:

1. The algorithm flow of MP and its MATLAB implementation
2. The algorithm flow of OMP and the realization of MATLAB
3. The difference between MP and OMP
4. The relationship between Schmidt orthogonality and Omp

One, MP (matching tracking) algorithm flow:

Second, the MP of MATLAB implementation:

%MP: Matching tracking algorithm%Dictionary: Super-Complete dictionary%x: Signal to be expressed% M =4; N =Ten;% Phi = RANDN (m,n); %Dictionary% fornn =1: N% phi (:, nn) = Phi (:, nn)/Norm (Phi (:, nn));%End% B = Randn (M,1); %signal function x=MP (Dictionary,x,iter) [M,n]=size (dictionary); residual= Zeros (M,iter); %residual matrix, which holds the residual residual after each iteration (:,1) = x; %Initialize residuals to XL= Size (Residual,2); %get the column pos_num of the residuals matrix= Zeros (1, L); %the column ordinal to hold each selection resi_norm= Zeros (1, L); %The 2-norm Resi_norm used to hold the residuals after each iteration (1) = norm (x); %because the previous initialized residuals are xiter_out= 1e-3; Iter_count=0; forMM =1: Iter%Iteration Exit CriteriaifResi_norm (mm) <Iter_out Break; End%Calculate the inner product scalarproducts of each column and the last residuals of dictionary= Dictionary'*residual (:, mm);%find the largest column in the inner product and its inner product value [Val,pos]=Max (ABS (scalarproducts)); %Update residuals residual (:, MM+1) = Residual (:, mm)-scalarproducts (POS) *Dictionary (:, POS); %calculates residuals of 2 norm (square and re-open radical) resi_norm (mm+1) = Norm (Residual (:, mm+1)); %Save the selected column ordinal pos_num (mm)=POS; Iter_count= Iter_count +1; end%the 2-norm curve of the residual error is plotted Resi_norm= Resi_norm (1: iter_count+1);p lot (resi_norm);%Display the selected dictionary Atom Pos_num= Pos_num (1: iter_count);d ISP (pos_num);%sparse coefficients (sparse representations) Dict=Dictionary (:, pos_num); Y_vec= (dict'*dict) ^ ( -1) *dict'*X;disp (Y_vec); Figure;plot (Y_vec);

Third, the OMP (orthogonal matching tracking) algorithm flow:

Four, the OMP matlab realization:

%MP: Matching tracking algorithm%Dictionary: Super-Complete dictionary%x: Signal to be expressed% M =4; N =Ten;% Phi = RANDN (m,n); %Dictionary% fornn =1: N% phi (:, nn) = Phi (:, nn)/Norm (Phi (:, nn));%End% B = Randn (M,1); %signal function x=OMP (Dictionary,x,iter) [M,n]=size (dictionary); residual= Zeros (M,iter); %residual matrix, which holds the residual residual after each iteration (:,1) = x; %Initialize residuals to XL= Size (Residual,2); %get the column pos_num of the residuals matrix= Zeros (1, L); %the column ordinal to hold each selection resi_norm= Zeros (1, L); %The 2-norm Resi_norm used to hold the residuals after each iteration (1) = norm (x); %because the previous initialized residuals are xiter_out= 1e-3; Iter_count=0; Aug_mat= []; forMM =1: Iter%Iteration Exit CriteriaifResi_norm (mm) <Iter_out Break; End%Calculate the inner product scalarproducts of each column and the last residuals of dictionary= Dictionary'*residual (:, mm);%find the largest column in the inner product and its inner product value [Val,pos]=Max (ABS (scalarproducts)); %the augmented matrix Aug_mat of least squares=[Aug_mat Dictionary (:, POS)]; %least squares projection proj_y= aug_mat* (Aug_mat'*aug_mat) ^ ( -1) *aug_mat'*x; %Update residuals residual (:, MM+1) = X-proj_y; %calculates residuals of 2 norm (square and re-open radical) resi_norm (mm+1) = Norm (Residual (:, mm+1)); %Save the selected column ordinal pos_num (mm)=POS; Iter_count= Iter_count +1; end%the 2-norm curve of the residual error is plotted Resi_norm= Resi_norm (1: iter_count+1);p lot (resi_norm);%Display the selected dictionary Atom Pos_num= Pos_num (1: iter_count);d ISP (pos_num);%sparse coefficient dict=Dictionary (:, pos_num); Y_vec= (dict'*dict) ^ ( -1) *dict'*X;disp (Y_vec); Figure;plot (Y_vec);

V. The difference between MP and OMP:

The difference between OMP and MP is essentially the residual update process: The Pem of the Omp minus is an orthogonal projection of em on the spanned space of the matrix φt of All selected atoms , The MP minus Pem is an orthogonal projection of em in the space spanned the atom Φm selected . Based on this, OMP ensures that the selected atoms are no longer selected.

Six, Schmidt (Schimidt) orthogonality and Omp1, Schmidt (Schimidt) orthogonal process:

The Upper [x,y] denotes the inner product of the vector, [x,y]=xty=ytx=[ x,y] . In the orthogonal formula of Schmidt bR can actually be written as:

score The reason the child can change this is because the actually a number, So [ x , y ] x = x [ x , y ]=  xx t y

2. The relationship between Omp and Schmidt (Schimidt) orthogonality:

Conclusion: The OMP decomposition process, in fact, is to schimidt the selected atoms sequentially, and then subtract the decomposition signal from the orthogonal atoms on the respective components can be residual. In fact (Formula 3) the process of seeking residual error is also in the Schmitt orthogonal.

3. Verifying the relationship between the process of OMP residual error and Schmidt orthogonality

%verifying the relationship between the process of OMP residual error and Schmidt orthogonality%Clc;clear;close All; M=4; N =Ten; Phi= Randn (m,n); %Dictionary fornn =1: N Phi (:, nn)= Phi (:, nn)/Norm (Phi (:, nn)); Endb= Randn (M,1); %Signal Res0= b; %initialization residuals for sparse signal B%OMP%Select a dictionary first atom C1= Phi'* RES0;% seeking matrix phi columns and B's inner product[VAL1,POS1] = MAX (ABS (C1)); %find the largest column in the inner product and its inner product value Phit= [Phi (:, POS1)]; %matrix Pphi by the combination of all selected columns= phit* (phit'*phit) ^ ( -1) *phit'; %orthogonal projection transformation matrix Omp_res1= Res0-pphi*res0; %OMP uses the last residual error minus the residual error in the orthogonal projection of the Phit column space Omp_resb= B-pphi*b; %omp with sparse signal B minus b orthogonal projection in phit column space%SCHIMIDTX= Phi (:, POS1); %Schimidt orthogonal first vector px= x* (x'*x) ^ ( -1) *x'; Smt_res1= Res0-px*b; % is actually b-px*b%test Norm (omp_res1-omp_resb) Norm (Omp_resb-smt_res1)%OMP%Select Dictionary second column C2= Phi'* OMP_RES1;[Val2,pos2] =Max (ABS (C2));p hit=[Phi (:, Pos1) phi (:, Pos2)]; Pphi= phit* (phit'*phit) ^ ( -1) *phit'; Omp_res2= omp_res1-pphi*Omp_res1;omp_resb= b-pphi*b;%Schimidty= Phi (:, Pos2)-Px*phi (:, Pos2); %Schimidt orthogonal second vector py= y* (y'*y) ^ ( -1) *y'; Smt_res2= Smt_res1-py*b; % is actually b-px*b-py*B, the last residuals minus B in the 2nd column orthogonal to the resulting Z projection%Testnorm (Omp_res2-omp_resb) Norm (Omp_resb-smt_res2)%OMP%Select Dictionary third column C3= Phi'* OMP_RES2;[VAL3,POS3] =Max (ABS (C3));p hit=[Phi (:, Pos1) phi (:, Pos2) phi (:, POS3)]; Pphi= phit* (phit'*phit) ^ ( -1) *phit'; Omp_res3= omp_res2-pphi*omp_res2; Omp_resb= b-pphi*b;%Schimidtz= Phi (:, POS3)-Px*phi (:, POS3)-Py*phi (:, POS3); %Schimidt orthogonal third vector pz= Z* (Z'*z) ^ ( -1) *z'; Smt_res3= Smt_res2-pz*b; % is actually b-px*b-py*b-pz*B, the last residuals minus B in the 3rd column orthogonal to the resulting Z projection%Testnorm (Omp_res3-omp_resb) Norm (Omp_resb-SMT_RES3)

Reference article:

http://blog.csdn.net/jbb0523/article/details/45099655

http://blog.csdn.net/jbb0523/article/details/45100351

Talking about the compression perception (19): MP, Omp and Schmidt orthogonality