A review of the ten concepts of stereo matching.

First, the concept

The stereo matching algorithm mainly establishes an energy cost function to estimate the pixel parallax value by minimizing the energy cost function. The essence of stereo matching algorithm is an optimal solution problem, by establishing reasonable energy function, adding some constraints, and using the optimization theory to solve the equation, which is also the solution of all ill-posed problems.

Second, the main stereo matching algorithm classification

1) based on the use of image representation of the different primitives, stereo matching algorithm is divided into:

A, regional stereo matching algorithm (dense disparity map can be obtained. Disadvantage: Affected by image affine distortion and radiation distortion, the size and shape selection of pixel-constrained window is difficult, the selection is too large, in the depth discontinuity, the disparity map will appear excessively smooth phenomenon, the selection is too small, the constraints on the pixel point less, the image information is not fully utilized, easy to produce mis-match. ）

B, feature-based stereo matching algorithm (sparse parallax map can be obtained, and the density disparity map can be obtained by the difference estimation.) It can extract local features such as point, line and polygon, and can also extract global features such as polygon and image structure. Disadvantages: Feature extraction is easily affected by occlusion, light, repetition texture, etc., and the difference value is computationally large.

C, based on the phase stereo matching algorithm (assuming that in the image corresponding point, its frequency range, its local phase is equal, in the frequency range of parallax estimation)

2) based on the use of optimization theory, stereo matching algorithm can be divided into:

A, local stereo matching algorithm

B, global stereo matching algorithm

Three, matching primitives (match primitive)

The matching primitives used in the current matching algorithm can be divided into two main categories:

1) Extract the measurement descriptors at all pixel points of the image

A, pixel grayscale value (simplest, direct, but must be obtained under the same lighting conditions)

B, Local area gray function (mainly using the derivative information of gray distribution in various sizes of different windows to describe the structure vector around the pixel point. ）

C, convolution image symbol (using various size operators and images for convolution, using gray gradient local maxima or minima as feature information, describe the entire image)

2) Image Features

A, over 0 points

B, Edge (because the edge is the image feature position, the change of gray value is not sensitive, edge is an important feature of image matching and descriptors)

C, corner Point (although it does not have a definite mathematical definition, but it is generally considered that the corner point, that is, two-dimensional image brightness changes sharply point or edge curve on the extreme point of curvature)

Four, region matching algorithm

The basic principle is given to a certain point in an image, select a sub-window in the neighborhood of the pixel point, in another image in an area, according to a certain similarity to determine the basis for the Sub-window image of the most similar to the sub-graph, and its matching sub-graph corresponding pixel point is the pixel's matching point.

The general simple area match encounters the following restrictions:

1) for weak textures or areas where duplicate textures exist, the results are not good

2) This algorithm is not suitable for the scene with drastic depth change

3) sensitivity to illumination, contrast and noise

4) The size of the subform is difficult to select

Five, feature matching algorithm

Feature matching algorithm is mainly based on geometric feature information (edge, line, contour, interest point, corner point and geometric primitive, etc.), to the geometric feature points for parallax estimation, so first to extract the feature points of the image, do
and using the Parallax value information of these feature points to reconstruct the three-dimensional space scene.

The main steps to match are: Image preprocessing, extracting feature, matching of feature points to get sparse disparity map, if we want dense disparity map, interpolation method is needed.

Six, global matching algorithm

Global stereo matching algorithm mainly uses the global optimization theory method to estimate parallax, establishes global energy function, and obtains the optimal parallax value by minimizing global energy function.

The result of global matching algorithm is accurate, but its running time is longer and is not suitable for running in real time. The main algorithms are graph cut (graph cuts), belief propagation (belief propagation), and dynamic programming.

Seven, local matching algorithm (individuals feel similar to the region, the angle is different)

The local optimization method is used to estimate the disparity value, the local stereo matching algorithm has SAD,SSD and other algorithms, and the global stereo matching algorithm, as well as the energy minimization method for parallax estimation, but in the energy function, only the data items, and no smoothing term.

It is mainly divided into three types: adaptive form stereo matching algorithm, adaptive weighted stereo matching algorithm and multi-form stereo matching algorithm.

Eight, stereo matching constraints

1) Polar Line constraints
2) Uniqueness constraints

3) Parallax Continuity constraints

4) Sequential consistency constraints

5) Similarity constraint

Ix. The criterion of similarity judgment

1) square sum of gray difference of pixel points, i.e. SSD

2) absolute value of gray difference in Pixel point and SAD

3) Normalization of cross-correlation, referred to as NCC

4) 0 mean cross correlation, i.e. ZNCC

5) Moravec non-normalized cross correlation, i.e. MNCC

6) Kolmogrov-smrnov distance, i.e. KSD

7) Jeffrey divergence

8) Rank transform (the gray value of the center pixel is replaced by the number of pixels in the window with a gray value less than the center pixel gray value)

9) Census transformation (is based on the center of the window Gray scale and the rest of the pixel gray value of the size of a string of code, bit length is equal to the number of pixels in the window minus one)

X. Parameters of Evaluation

Stereo matching algorithm is a morbid problem, generally through the establishment of energy functions, the use of the minimization of energy functions, and some constraints, the optimization of the theoretical method to solve the equation.

Recognized quantitative evaluation parameters are: RMS error (root-mean-squared) and false match rate (percentage of bad matching pixels)