Several calculation methods of stereo video disparity map

1, Absolute difference value method
The absolute difference graph is calculated as follows:
da=| l-r|
, L, R, and Da represent the luminance values of the left view, the right view, and the corresponding absolute difference graph, respectively. The absolute difference graph is not a strict disparity map, but its calculation method is the simplest, the speed is fast, and the result can be used as a reference.

2. Figure Cutting Method
Global matching algorithm is a kind of main algorithm of stereo matching. Hai Tao and other people put forward a new framework of stereo matching algorithm, the framework of the algorithm is not a window unit, but a global energy function to characterize the stereo image matching situation, and try to minimize the global energy function, so as to obtain a stereoscopic image of the parallax image, which is the global stereo matching algorithm framework. The global stereo matching algorithm can avoid the common errors in many regional stereo matching algorithms, and has a good performance in the low texture region. Graph cutting method is a representative algorithm of global matching algorithm, Boykov and so on, the idea of graph cutting method is applied to stereo matching algorithm first. This study proposes two kinds of optimization ideas for the parallax maps of α, β-exchange and α-expansion, and optimizes the area of disparity equal to a certain value in all parallax maps at once. The algorithm adopts the idea of graph cutting method, according to the correlation degree of neighboring points and the matching degree between neighboring point and hypothesis disparity, re-distributes and adjusts the relationship between the original pixel and parallax value, and transforms the stereo matching problem into solving the problem of maximum flow minimum cut, until the global energy function is minimized.
In the configuration of Open source code of graph cutting, there are KZ1 methods with good effect and speed (KZ1, KZ2 and Bvz total three methods), the Parallax range is set to [ -30,30] and the number of iterations is 3 times. Graph cut input and output image format is ppm format, when reading YUV file, the YUV file needs to convert each frame into RGB format, and add the header information "P6", "width High" and "255", input ppm format image, Storage parallax video, you need to skip the PPM format image header information, Convert RGB information to YUV format. The conversion formula between YUV and RGB is as follows:
y= 16+0.183XR+0.614XG+0.062XB
U=128+0.101xr+0.339xg+0.439xb
V=128+0.439xr+0.399xg+0.040xb

R=1.164x (y-16) +1.793x (v-128)
G=1.164x (y-16) -0.213x (u-128) -0.533x (v-128)
B=1.164x (y-16) +2.112x (u-128)

The parallax map computed by the graph cutting method is the most accurate, but it is very slow, it takes a few minutes to complete the calculation of the single frame disparity map, and it is not suitable for the objective evaluation method of stereoscopic video quality, but the result can be used as a reference.

3. Region matching method based on fixed window
The simplest form of region matching is to divide the whole image into many sub-regions, and then measure the luminance information or the gray distribution information of all the sub-regions. It creates a window centered on the matching point of the Datum map, and when the window is fixed, the region matching method is based on the fixed window. Using the gray value distribution in the window to characterize the pixel, and then searching for such a pixel in the alignment graph, creating the same window as the center, and characterizing it with the gray value distribution of its neighborhood pixels, and searching for the best match according to the matching criterion constraint, the similarity between the two must satisfy certain threshold condition. The essence is to use the correlation degree of gray information between local windows to quantify the image to many image blocks and determine the corresponding region. The matching process is shown in the figure:
　　　　　　　　　　
In the configuration of the region matching method, the fixed window size is 11x11 and the Parallax range is set to [ -64,64]. The accuracy of the parallax map computed by the method of region matching based on fixed window is very poor, but its speed is very fast, and the result can be used as reference.

4, left and right consistency constraint principle
A pixel can only correspond to one parallax value, which is called the uniqueness constraint in stereo matching. Since a pixel does not have more than 2 parallax values, this constraint can also be described as the reference image and the target image match points should be one by one corresponding, that is, left and right consistency constraints.

The Masked area is imaged only on one monocular or a single camera, the reference image of a point in the target image can not find the corresponding matching point, if only to calculate the parallax map of the reference image, it is possible to have a pixel point has more than 2 parallax value to meet the threshold conditions, Thus violating the left and right consistency constraints, resulting in a matching error.
If the reference image and the target image position is reversed, the target image can be based on the corresponding point in the reference image, and a new match is made, which is called the inverse matching in stereo matching. Accordingly, the general matching process is called forward matching in stereo matching. If the three-dimensional image pairs are matched and matched in two processes, it is possible to find the wrong matching points violating the left and right consistency constraints, which is called left-right consistency detection. The accuracy of the parallax map computed by using the fixed window-based region matching method is very poor, but after the left and right consistency detection, the inaccurate part is eliminated, the accuracy of the depth information obtained is improved, and the speed of the regional disparity map detected by the left and right consistency is also very fast.