Introduction to Binocular vision algorithm

http://blog.csdn.net/u010784534/article/details/50437612

Reprinted from: Http://blog.sina.com.cn/s/blog_4a540be60102v44s.html 1. Introduction to Binocular vision algorithm

1.1. Introduction to Binocular Vision

Binocular vision is widely used in robot navigation, precision industrial Measurement, object recognition, virtual reality, Scene reconstruction, surveying field.

What is binocular vision?

Binocular vision is a method of simulating the principle of human vision, using the computer passive sensing distance. Observing an object from two or more points, acquiring the image at different viewing angles, according to the pixel matching relation between the images, calculates the offset between pixels by triangulation principle to obtain three-dimensional information of the object.

The depth of field information of the object can be calculated to calculate the actual distance between the object and the camera, the object 3-dimensional size, and the actual distance between two points. At present, there are many research institutions to identify 3-dimensional objects, to solve the 2D algorithm can not handle occlusion, attitude change problems, improve the object recognition rate.

1.2. Algorithm General flow

Figure 1 Binocular Vision algorithm Flow

1.2.1. Offline calibration

The purpose of calibration is to obtain the camera's internal reference (focal length, image center, distortion coefficient, etc.) and external parameters (R (rotation) matrix T (translation) matrix for its two cameras). At present, the methods commonly used for the Zhang Zhengyu calibration method, OPENCV and MATLAB are implemented. However, in order to obtain a higher calibration accuracy, the use of industrial-grade (60*60 lattice) glass panel effect will be better. And some people also recommend using MATLAB, because the accuracy of the visualization will be better, and the results of Matlab saved as XML,OPENCV can also be read directly, but the steps with respect to OpenCV a bit of trouble. This is the connection of the MATLAB calibration Toolbox: http://www.vision.caltech.edu/bouguetj/calib_doc/

The steps are:

(1) Left camera calibration, get internal and external parameters.

(2) The right parameter camera calibration obtains the external parameter.

(3) Double target, get the translation rotation relation between camera.

Figure 2 MATLAB Binocular Vision Calibration

1.2.2. Binocular correction

The goal of correction is to obtain a reference graph and a target graph, only the difference in the x direction exists. Improve the accuracy of parallax calculations.

The correction is divided into two steps

L Distortion Correction

Fig. 3 Distortion correction

L CONVERT the camera into a standard form

Figure 4 Converting to standard form

Because of the corrective part, the location of all points of the image will be recalculated, so the resolution of the algorithm processing time is greater, and generally need to process two images in real time. Moreover, this algorithm has a high degree of parallelism, it is recommended to use Ive for hardening, similar to the acceleration mode in OPENCV, first get map map, and then parallelize using map map to get pixel location again. The corrective function in OpenCV is cvstereorectify.

1.2.3. Binocular Matching

Binocular matching is the core of binocular depth estimation, it has been developed for many years, and there are many algorithms, the main purpose is to calculate the relative matching between the reference graph and the target graph, mainly divided into local and nonlocal algorithms. Generally there are several steps below.

L Matching Error calculation

L Error Integration

L parallax Graph calculation/optimization

L Correction of Parallax map

A general local algorithm, using a fixed-size or non-fixed-size window, to calculate the optimal match position for the row in which it is located. For the simplest local way, find the best corresponding point position, left and right view x coordinate position difference is disparity map. In order to increase the noise, the robustness of the illumination can be matched with a fixed window, or the image can be used after LBP transformation. The general matching loss calculation functions are: SAD,SSD,NCC and so on. Maximum parallax can also be used to limit the maximum search range, and the integration chart and box filter can be used for accelerated calculations. At present, the better local matching algorithm is based on the guided filter's Binocular matching algorithm using box filter and integral graph, the local algorithm is easy to parallelize, the computation speed is fast, but for the region with less texture, the image is segmented, and the image is divided into the area of texture richness and sparse texture. Adjust the matching window size, the texture sparse use small window, to improve the matching effect.

Fig. 5 Parallax calculation based on linear search

The non-local matching algorithm, which considers the task of searching parallax to minimize a certain loss function based on all binocular matching pairs, obtains the best disparity relation by finding the minimum value of the loss function, and focuses on solving the problem of the matching of the uncertain region in the image, which is mainly dynamic programming (dynamical programming), Trust propagation (Blief propagation), Graph cut algorithm (graph cut). At present, the best effect is also the graph cutting algorithm, the graph cut algorithm provided in OPENCV time-consuming is very big.

Graph cutting algorithm is mainly to solve the problem that the dynamic programming algorithm cannot fuse the horizontal and vertical direction continuity constraints, and the matching problem is regarded as the use of these constraints in the image to seek the minimum cutting problem.

Figure 6 is based on graph cut (left), dynamic planning (middle), correlation calculation (right) effect.

Because the global energy minimization is considered, the non-local algorithm is generally time-consuming and not very good to use hardware acceleration. But for the occlusion, the texture sparse situation solves better.

After the matching points are obtained, it is common to detect and determine the matching points with high confidence level through the consistency of the left and right lines. It is similar to the idea that the front-to-back optical flow is matched only by the point of the consistency test of the left and right sight. This can also be found because of the occlusion, noise, the wrong match to get points.

So how do we get parallax and depth information?

Regarding the post-processing of parallax map, the median filtering method is used to replace the gray value of the current point with the median value of the neighborhood pixels, which can remove the salt and pepper noise well. Outliers that fail due to noise or weak texture matching can be removed.

Introduction to Binocular vision algorithm