Sharing a binocular Ranging project _ binocular ranging

Tools and platforms to use

Opencv2.4.11

VS2010

Matlab

Reference papers and Blogs

Research of object depth information extraction system based on binocular Stereo Vision _ Liouville (Master thesis)

http://blog.csdn.net/scyscyao/article/details/5443341

Learning OpenCV 11, Chapter 12

The connection of the camera

Technical parameters (final effect)

———————————————------———————————————————

This article is mainly about implementation, and does not describe mathematical reasoning in detail.

Distance measurement principle

The principle of ranging is the use of triangular principles (of course this is for planar models, the three-dimensional model is represented by a matrix), the principle of the following figure:

The stereo model is represented by the following matrix relationship:

Stereo Calibration

The reason for calibration: the first is to determine the camera's coordinate origin, focal length, pixel and physical size ratio; The second is to eliminate the effect of lens distortion.

In OpenCV, two matrices are used to represent

Internal parameter matrix (Intrinstic_matrix):

Distortion parameters

Method and principle of calibration

General use of the board calibration hair, detailed principles can refer to Dr. Zhang Zhengyu's thesis

Here is a brief introduction to the principle, as shown in the following illustration:

The coordinates of the real world Q and the coordinates of its projection to the corresponding point of the image, using homogeneous coordinates to represent the following

The mapping of an object plane to an image plane can be expressed by using the following single response matrix H:

Q is known as the real board coordinates, and the image point Q can be found by the corner lookup method, so that according to the knowledge of linear algebra, given enough points, we can get many equations, and then we can solve the unknown internal parameters. Because the board of the market is very expensive, it is recommended to use A4 paper to print and paste to the rigid plane.

Stereo correction

In order for the two cameras to be mathematically aligned (rather than physically aligned) to the same observation plane, a rotational matrix, the R-shift matrix T, is defined, and the two matrices can be computed by known matching points, just as a single response matrix is obtained.

The following diagram shows the effect of correction

Stereo Matching

Sad matching algorithm

The method is centered on the source matching point of the Zootia image, define a window D, its size (2m+1) (2n+1), statistics its window's gray value, and then in the right image to gradually calculate the gray and the left window of the difference, the last search to the smallest difference in the area of the center pixel is the matching point.

Pseudo Code Calibration

nimages=10
squarewidth=200//Checkerboard square size 200mm 
squarewidth=200//The 
World coordinate value for the checkerboard corner point for
	(i = 0; i < nimages ; i++)
	{
	   n = 0;
	   for (j = 0; J < Boardsize.height; J +) for
		(k = 0; k < boardsize.width k++)
		objectpoints[i][n++] = CV::P o int3f (j*squarewidth, k*squarewidth, 0);
	}
Gray
Cvtcolor (left_frame_rgb,left_frame_gray,cv_bgr2gray);
Binary
threshold (left_frame_gray,left_frame_gray,thresholdness,255,cv_thresh_binary); 
To find the corner point, the corner coordinates are saved in the left_corners
bool Left_found =findchessboardcorners (Left_frame_gray,chessboardsize,left_ corners);
Draw the corner point
drawchessboardcorners (Left_frame_match, Chessboardsize, Cv::mat (left_corners), left_found);

From the above checkerboard calibration can obtain two important data point matrix (data type is Cv::mat) objectpoints left_corners

Double-Objective fixed 
stereocalibrate (
		objectpoints,  //checkerboard actual coordinate
		imagePoints1,  //left image corresponding Checkerboard corner coordinates
		ImagePoints2,  //<span style= "font-family:arial, Helvetica, Sans-serif;" > left image corresponding to the chessboard corner coordinates </span>
		Cameraparams1.cameramatrix,  //Camera Parameters
		Cameraparams1.distortioncoefficients,//Left camera distortion parameter
		Cameraparams2.cameramatrix,//   right camera internal parameters
		Cameraparams2.distortioncoefficients,//Right camera distortion parameter
		imagesize,//    image size
		rotation,    //rotation matrix
	        Translation,//translational concentration
		essential,//   Eigen Matrix
		Foundational,//base concentration
		Cv::termcriteria (cv::termcriteria :: Count+cv::termcriteria::eps, 1e-6,
		stereoparams.flags +
		Cv_calib_fix_k3 + cv_calib_fix_k4 + CV_CALIB _fix_k5
		);

Because the stability of the stereo calibration function provided by OPENCV is not very good, and the precision is inaccurate, the three-dimensional calibration of MATLAB is used in this experiment, which can be referred to
Stereo correction

<pre name= "code" class= "CPP" >

Stereorectify (
		Cameraparams1.cameramatrix,
		cameraparams1.distortioncoefficients,
		Cameraparams2.cameramatrix,
		cameraparams2.distortioncoefficients,
		imagesize,
		rotation,
		Translation,
		r1,r2, P1, P2, Q, 
		1024,
		Alpha, 
		imagesize, &roi1, &roi2
		);
	
		A mapping matrix Initundistortrectifymap (Cameraparams1.cameramatrix) for generating stereo correction needs
		,
		Cameraparams1.distortioncoefficients,
		R1, P1, 
		imagesize,
		cv_16sc2,
		remapmatrixs.mx1, REMAPMATRIXS.MY1);

	Initundistortrectifymap (
		StereoParams.cameraParams2.cameraMatrix,
		StereoParams.cameraParams2.distortionCoefficients,
		R2, P2, 
		stereoparams.imagesize,
		cv_16sc2,
		remapmatrixs.mx2, Remapmatrixs.my2);

Stereo Matching

<pre name= "code" class= "CPP" >

Cvtcolor (Frameleft, Img1proc, Cv_bgr2gray);

Cvtcolor (Frameright, Img2proc, Cv_bgr2gray);	
Remap (Img1proc, Img1remap, Remapmatrixs.mx1, remapmatrixs.my1, cv::inter_linear);

Remap (Img2proc, Img2remap, REMAPMATRIXS.MX2, Remapmatrixs.my2, cv::inter_linear);
Expand the original picture Cv::mat Img1border, Img2border;
Copymakeborder (Img1remap, img1border, 0, 0, m_bm.state->numberofdisparities, 0, ipl_border_replicate);

Copymakeborder (Img2remap, img2border, 0, 0, m_bm.state->numberofdisparities, 0, ipl_border_replicate);
Calculate view difference Cv::mat Dispborder;
M_BM (Img1border, Img2border, Dispborder); Crop picture Cv::mat M_calib_mat_mask_roi = Cv::mat::zeros (StereoParams.imageSize.height, StereoParams.imageSize.width, CV
_8UC1);
Cv::rect Roi (0,0,640,480);
Cv::rectangle (M_calib_mat_mask_roi, Roi, Cv::scalar (255),-1);
Cv::mat disp;	
	

DISP = Dispborder.colrange (m_bm.state->numberofdisparities, img1border.cols);
Cv::mat disparity;

Disp.copyto (disparity, m_calib_mat_mask_roi); if (disparity.depth ()!= cv_8u) {diSparity.convertto (disp8u, cv_8u, 255/(m_bm.state->numberofdisparities*16.)); ///Convert two-dimensional parallax map to three-dimensional coordinate graph reprojectimageto3d (DISP8U,OUT_3D,REMAPMATRIXS.Q);