Parallax matching of images (Stereo Matching)

Here we compare our own computed parallax map with the given parallax map to compare the quality of the parallax map we get, and the value returned by my parallax graph is the computed parallax multiplied by 3 , so I am not two when I am calculating a difference greater than 1 , but more than 3, because two images are multiplied by 3 , so to be greater than 3, I passed in the parameter is two images of the matrix, Because I wrote a script that ran all the samples, and the image was read out in the script.

<span style= "FONT-SIZE:18PX;" >function [Percentnumberbadpixels] = Percentbadpixels (mydisparitymap,groundtruthdisparitymap) Mydisparitymap = Double (mydisparitymap); groundtruthdisparitymap = double (groundtruthdisparitymap); [Rmydisparitymap, Cmydisparitymap] = size (mydisparitymap); numbadpixels = 0;for i = 1:rmydisparitymap for    j = 1:Cmydis Paritymap        if (ABS (Mydisparitymap (I,J)-Groundtruthdisparitymap (I,J)) > 3)              numbadpixels = numbadpixels+1;        End    Endendpercentnumbadpixels = numbadpixels/(rmydisparitymap*cmydisparitymap*1.0);p ercentnumbadpixels = Floor (percentnumbadpixels*100); str = strcat (NUM2STR (percentnumbadpixels), '% ');p ercentnumberbadpixels = Str;end</span >

SSD:

Left parallax Map

Right parallax Map

Implementation Details:

The input image is a four-bit image, but these calculations are calculated based on strength (intensities), in matlab The function of turning color graph into gray figure is essentially rgb2gray to its intensity grayscale image.

Principle:

For the same image, the image seen in the left eye is not the same as the position of the image seen in the right eye, for example, for the position of the image seen in the right eye at the same position in the front of the position of the image seen in the left eye, so that the left Parallax map and right disparity map search methods, for left parallax map

Since its position is far beyond the right eye, it is possible to find the closest part in the same physical location as the right eye image, which is to subtract D.

Similarly, the right parallax map is just the opposite, plus D.

Implementation Details:

1) First convert the image through intensities to grayscale Image:

2) Set the value of the search depth D and patch values, and then the image to complement the boundary, so that you can start from the original image of the first pixel to match, I use the complement of the boundary method and the same principle used to do the filter, I'm all about the value around.

3) for each pixel in the left or right image (i,J) as the center of the patch in another left or right image to find the closest The search depth for patch section D, specifically:

For each non-moving in the template imagePatch, put thePatcheach pixel in the inside corresponds to a different target graph .Patch(plus search depthD) on the pixel dot do the squared sum and then add up, each oneDcorresponds to aPatch, each onePatchcorresponds to a value (sum), and then find out the depth of the searchDwithin the range of the smallestsum, thus obtaining the smallestsumthe search depthDwhen the boundary is encountered, it is handled with judgment, and is implemented as follows:

If it is the right parallax map, the above principle and introduction, to add D to search

NCC

Left parallax Map:

Right parallax Map:

Implementation Details:

The image is converted to grayscale and the boundary processing is the same as SSD, the formula realizes:

i j patch patch part of the search depth d

1) for each patch in the template image that does not move , the patch that corresponds to each pixel inside the patch and the other target graph (plus the search depth d ), the corresponding pixel points are multiplied and added separately, the patch in the template diagram and the patch of the target map The pixel values in the square are added separately and then multiplied by two values.

2) Each D corresponds to a patch, and each patch corresponds to a value (ncctemp), Then we find the largest ncctempin the range of the search depth D , thus obtaining the search depth of the largest ncctemp . The value of D, which is handled with judgment when the boundary is encountered

The difference between NCC and SSD:

NCC has strong illumination resistance, and SSD has weak light resistance

such as: ADD A small constant amount of intensity (e.g.) to all right eye images, and re-run the above, methods. Analyze How the intensity change affects the results (i.e. the quality) of the of the methods. Explain in which ways, the NCC is a better matching cost than SSD.

I've added a constant to view5 here.

NCC results are as follows:aloe disp1 Bad point rate is 24%

VIEW5 strength no plus is the bad point rate is:24%

SSD results are as follows:aloe disp1 Bad point rate is 38%

VIEW5 strength no plus is the bad point rate is:26%

Analysis:

The formula above can be SSD is based on the two images on the patch d is a matching part, but in the photo is, will be affected by a number of factors, such as the intensity of light, such as the left eye and the right eye to see the intensity of the image is not the same, if the right eye to see the intensity of the image increased 10 patch d

For NCC , the pixels are multiplied by two patches , and the denominator is The square of the pixel value in each patch, and the sum is multiplied, Even if the brightness of the right eye is added, the values of the numerator and denominator both increase at the same time, and the value of the increase is almost the same, and the increased value is nearly 1, so the resulting result is essentially the same as the result of theright eye intensity without a plus . , there's not much change.

NCC is better than SSD If the left and right eye images are disturbed by light intensity .

nccdispl.m

function [Output_img] = NCCDISPL (leftimg,rightimg) [Nrleft,ncleft,nleft] = size (leftimg); [Nrright,ncright,nright] =size (rightimg);d isparitymap = zeros (nrright,ncright); if (Nleft > 2) leftimg = Rgb2gray (lef    TIMG); endif (Nright > 2) rightimg = Rgb2gray (rightimg); rightimg = Rightimg+10;endd = 70;winsize = 11;disparitymap = Zeros (nrright,ncright); leftimg = double (leftimg); rightimg = d Ouble (rightimg); leftimg = Supplyborder (leftimg,winsize); rightimg = Supplyborder (rightimg,winsize); win = (winsize-1)        2;for i = 1+win:nrright+win for j = 1+win:ncright+win min =-9999999;            For k = 0:d numerator = 0.0;            Powerrightwin = 0.0;            Powerleftwin = 0.0; For A=-win:win for b =-win:win if j+b-k > 0 numerator = Numer                        ator+ (leftimg (i+a,j+b) *rightimg (i+a,j+b-k));                        Powerleftwin = powerleftwin+ (leftimg (i+a,j+b) *leftimg (i+a,j+b)); PowerrighTwin = powerrightwin+ (rightimg (i+a,j+b-k) *rightimg (i+a,j+b-k));            End End End ncctemp = numerator/(sqrt (Powerrightwin*powerleftwin));                if (min < ncctemp && j+b-k > 0) min = ncctemp;            RECORDK = k;    End End Disparitymap (i-win,j-win) = RECORDK; Endenddisparitymap = Uint8 (3*disparitymap); output_img = disparitymap;% imshow (disparitymap);% Imwrite (Disparitymap, strcat (' C:\Users\samsung-\Desktop\output\nccdisp1.png ')); end

</pre><pre name= "code" class= "plain" ><pre name= "code" class= "plain" style= "font-size:18px;" >ssddispl.m

<pre name= "code" class= "plain" >function [output_img] = SSDDISPL (leftimg,rightimg) [Nrleft,ncleft,nleft] = size ( LEFTIMG);  [Nrright,ncright,nright] =size (rightimg); if (Nleft > 2) leftimg = Rgb2gray (leftimg); EndIf (Nright > 2) rightimg    = Rgb2gray (rightimg); rightimg = Rightimg+10;enddisparitymap = Zeros (nrleft,ncleft);d = 70;winsize = 11;leftimg = Double (leftimg); rightimg = Dou BLE (rightimg); leftimg = Supplyborder (leftimg,winsize); rightimg = Supplyborder (rightimg,winsize); win = (winsize-1)/2;        For i = 1+win:nrright+win for j = 1+win:ncright+win max = 99999999;            for k = 0:d sum = 0.0; For A=-win:win for b =-win:win if j+b-k > 0 temp = leftimg (i                         +A,J+B)-rightimg (i+a,j+b-k);                         temp = temp*temp;                    sum = sum+temp; End End if (Max > Sum && j+b-k >0) max = sum;            Mind = k;    End End Disparitymap (i-win,j-win) = mind; Endenddisparitymap = Uint8 (3*disparitymap); output_img = disparitymap;% imshow (disparitymap);% Imwrite (Disparitymap, strcat (' C:\Users\samsung-\Desktop\output\ssddisp1.png ')); end

<pre name= "code" class= "plain" >nccdispr.m

function [Output_img] = NCCDISPR (leftimg,rightimg) [Nrleft,ncleft,nleft] = size (leftimg);  [Nrright,ncright,nright] =size (rightimg); if (Nleft > 2) leftimg = Rgb2gray (leftimg); EndIf (Nright > 2) rightimg    = Rgb2gray (rightimg); rightimg = Rightimg+10;endd = 70;winsize = 11;disparitymap = Zeros (nrleft,ncleft); leftimg = double (leftimg); rightimg = Dou BLE (rightimg); leftimg = Supplyborder (leftimg,winsize); rightimg = Supplyborder (rightimg,winsize); win = (winsize-1)/2;        For i = 1+win:nrleft+win for j = 1+win:ncleft+win min =-999999;             For k = 0:d numerator = 0.0;             Powerrightwin = 0.0;            Powerleftwin = 0.0; For a =-win:win for b =-win:win if j+b+k <= ncleft+2*win num                        Erator = numerator+ (rightimg (i+a,j+b) *leftimg (i+a,j+b+k));                        Powerrightwin = powerrightwin+ (rightimg (i+a,j+b) *rightimg (i+a,j+b)); Powerleftwin = powerleftwin+ (leFtimg (i+a,j+b+k) *leftimg (i+a,j+b+k)); End End End ncctemp = numerator/(sqrt (powerrightwin*powerleftwin            ));                if (min < ncctemp && j+b+k <= ncleft+2*win) min = ncctemp;            RECORDK = k;    End End Disparitymap (i-win,j-win) = RECORDK; Endenddisparitymap = Uint8 (3*disparitymap); output_img = disparitymap;% imshow (disparitymap);% Imwrite (Disparitymap, strcat (' C:\Users\samsung-\Desktop\output\nccdisp5.png ')); end

</pre><pre name= "code" class= "plain" ><pre name= "code" class= "plain" style= "font-size:18px;" >ssddispr.m

<pre name= "code" class= "plain" >function [output_img] = SSDDISPR (leftimg,rightimg) [Nrleft,ncleft,nleft] = size ( LEFTIMG);  [Nrright,ncright,nright] =size (rightimg); if (Nleft > 2) leftimg = Rgb2gray (leftimg); EndIf (Nright > 2) rightimg    = Rgb2gray (rightimg); rightimg = Rightimg+10;enddisparitymap = Zeros (nrleft,ncleft);d = 70;winsize = 11;leftimg = Double (leftimg); rightimg = Dou BLE (rightimg); leftimg = Supplyborder (leftimg,winsize); rightimg = Supplyborder (rightimg,winsize); win = (winsize-1)/2;        For i = 1+win:nrleft+win for j = 1+win:ncleft+win max = 999999999;            for k = 0:d sum = 0.0;  For A=-win:win for b =-win:win if j+b+k <= Ncleft+2*win Temp                         = Rightimg (i+a,j+b)-leftimg (i+a,j+b+k);                         temp = temp*temp;                    sum = sum+temp; End End if (Max > Sum && j+b+k <= ncleft+WIN) max = sum;            Mind = k;    End End Disparitymap (i-win,j-win) = mind; Endenddisparitymap = Uint8 (3*disparitymap); output_img = disparitymap;% imshow (disparitymap);% Imwrite (Disparitymap, strcat (' C:\Users\samsung-\Desktop\output\ssddisp5.png ')); end

Parallax matching of images (Stereo Matching)