Summary of some stereoscopic vision

These two days I have been studying things related to stereoscopic vision, using a combination of opencv2.0 + vs2005. Opencv is a very powerful library of computer vision. With it, many complex algorithms can be implemented through a few simple functions. Of course, the premise is that you have to configure various variables and matrices that the function needs to call. Too many variables need to be prepared to call some complex opencv functions, which often account for half or more of the program. Here, I feel that it is not enough to simply learn how to call a function. We still need to thoroughly study the principle and do not want to understand the function source program (this is too big, it is estimated that only the experts can understand it), but at least the algorithm should be understood. Many functions of opencv are based on Algorithms in some excellent papers, using Google's academic search for these articles is still rewarding.

After talking about so much nonsense, I would like to summarize some of the results of the two-day study. Opencv2.0 has a function cvstereocalibrate, which is used to calibrate the relative positions and rotation angles of two cameras. Of course, to call this function requires a lot of preparation in advance. First, opencv uses the method in Zhang zhengyou's paper to calibrate the camera. Therefore, a black and white board is required. You only need to print one and paste it on the plane. I used a chessboard under the opencv installation directory, which is 7x7. Opencv provides cvfindchessboardcorners and this function to obtain the corner position of the board in the image for next calibration. In practical application, I found that it is better to use a board with different lengths and widths. The cvfindchessboardcorners function of opencv sometimes gets wrong directions on the X axis and Y axis, leading to a mess in the subsequent calibration results. So I used a MATLAB toolbox to calibrate the camera. Here I have referred to the research results of many experts, mainly referring. After the two cameras are calibrated using the MATLAB toolbox, the internal parameters, distortion parameters, translation vectors and rotation vectors between the two cameras can be obtained.

Here, the foucal length is the focal length of the camera in the X and Y directions, that is, to learn the FX and FY values in Chapter 11th of opencv; principal point refers to the camera offset, that is, CX and Cy; skew is not used here; distortion refers to the radial and tangent distortion of the camera, that is, the five parameters K1, K2, P1, P2, K3, the order is recorded in the order described by opencv. The four parameters fx fy cx xy here constitute the first two parameters of the cvstereorecloud function (of course, Left camera and right camera). The specific structure is shown in figure.

The K1, K2, P1, P2, and K3 parameters constitute the third and fourth parameters of the cvstereorecloud function. It is pointed out that we use the MATLAB toolbox to calibrate an om vector, which is the result of the rotation matrix transformation through Rodrigues. Therefore, we need to first convert this vector into a rotation matrix using cvrodrigues. However, the translation vertor is the displacement matrix of the right camera relative to the Left camera. This only needs to be copied directly to the XML file of opencv. In this way, you can call the cvstereorecloud function to obtain all the parameters to be prepared, next, we will use this function and cvinitundistortrectifymap and cvremap to get the two graphs aligned online.

Next, you only need to use the cvfindstereocorrespondencebm function to obtain the depth information. Note that this function requires a cvstereobmstate struct to be used as the preprocessing filter parameter in addition to the left and right images with two polar lines aligned. For specific configuration, see configure. After cvfindstereocorrespondencebm is called, normalize the depth information to 0 ~ 255 is displayed. Here, I also used a function of chenyusiyuan's big guy to use a pseudo-color image to display depth information. The final depth image.

Here we can see that there are still a lot of noise and no matching places, but in general, the depth of the obstacle is basically restored. Here, the cvstereobmstate parameter I used is

Bmstate-> prefiltersize = 13;
Bmstate-> prefiltercap = 13;
Bmstate-> sadwindowsize = 19;
Bmstate-> mindisparity =-32;
Bmstate-> numberofdisparities = 128;
Bmstate-> texturethreshold = 10;
Bmstate-> uniquenessratio = 20;
Bmstate-> specklewindowsize = 13;

To sum up, although the calibration functions provided by opencv are easy to call, they are less intelligent and prone to incorrect corner points. Although the MATLAB toolbox is troublesome, it can be manually specified to find some locations, so the calibration success rate is high, using the calibrated parameters, it is easy to obtain satisfactory results in the subsequent depth information restoration. Therefore, I prefer to use the MATLAB toolbox to calibrate the two cameras. At the same time, the MATLAB toolbox also has some other functions, so it is more flexible.

I will summarize it here and add new ideas.