Seven, three-dimensional calibration and stereo correction

Seven, three-dimensional calibration and stereo correction

In this blog post, let's get to know how Zhang's calibration is transitioning to stereo calibration. Here the main binocular stereoscopic vision analysis. For Binocular stereoscopic vision, we have two cameras. They are like a pair of eyes, looking at the world from different directions. The parallax of the images in the two eyes gives us a three-dimensional understanding of the world.

So, to know Parallax, we should first know the relative position relationship between two cameras in binocular vision system. We can calibrate the two cameras simultaneously and get the rotation matrix and the matrix of the two relative coordinates. The process of obtaining these two matrices is the process of stereoscopic calibration. That is: From Zhang's calibration to three-dimensional calibration.

The rotation matrix and the matrix between the two cameras can be calculated from the following formula:

where r is the rotation matrix between the two cameras, T is the translation matrix between the two cameras. The RR is the rotation matrix of the relative calibration obtained by the right camera through the tensor calibration, and the TR is the translation vector of the relative calibration obtained by the right camera through the tensor calibration. The RL is the rotation matrix of the relative calibration objects obtained by the left camera through the tensor calibration, and the TL is the translation vector of the relative calibration object obtained by the left camera through the tensor calibration.

The two formulas can be deduced by mathematical substitution. But maize felt that there were obvious physical meanings and simple computational formulas for spatial processes. The physical space imagination is more vivid than the substitution of pure algebra. And it helps us to figure out the whole physical process.

In the corn eye, these two formulas are:

For R, first, the left camera coordinate system is shifted to the right camera coordinate system (i.e., the two coordinate systems are coincident at the far point). Then multiply the two rotation matrices under the same reference system to indicate that world coordinates are rotated to the right to the RR and then to the left by the RL. So the rotation after two rotations is the rotation matrix R required to rotate the camera to the left camera.

For T, it is easier to understand, first with R on the left coordinate system rotation, the right and left two cameras to parallel, and then directly translate the vector subtraction, that is obtained. The translation vector t between the two cameras.

The above-calculated R and T are the parameters obtained by the stereo calibration.

So the use of the three-dimensional calibration obtained parameters, the next step we should do. The answer is: Stereo correction.

Before introducing the specific method of stereo correction, let's look at why we need to do stereo correction.

Because when two image planes are fully coplanar, the computation of stereo parallax is the simplest. However, in the reality of Binocular stereo vision system, there is no full coplanar line alignment of the two camera image plane. So we're going to do a stereo correction. The purpose of stereo correction is to correct two images of the actual non-coplanar line alignment to the Coplanar line alignment. As shown in the diagram below. (Coplanar line alignment refers to: Two camera image plane on the same plane, and the same point projected to two camera image plane, should be in two pixel coordinate system of the same row)

With the above bedding, let the corn give you a description of the three -dimensional calibration based on the parameters obtained by the stereo correction of the mathematical principle, or the geometrical principle. But the corn here, mainly to share with you, so corrected the physical meaning.

Stereo correction should take two steps:

1. Pull the two image planes back to the same plane.

This step, how to do it. Corn believes that after so many bedding, we should have been in the mind. This is a simple step. Can this: Two plane in the direction of the difference between a rotation matrix R, then we let two cameras rotate half, but note that the rotation of the two should be reversed. As shown in the following formula:

The above formula is corn according to the physical meaning of its own summary. The RL and RR represent both the left and right cameras to achieve the desired rotation matrix for the coplanar surface. RL, RR the same degree, but the opposite direction of rotation. After the camera has undergone such a rotation, the two are already coplanar. Everyone should understand.

2, rotate the image so that peer alignment

Corn here first to show you a, correct the success of the map bar.

You can look at the same as the primary school when you write a speech composition, observe the figure before and after the correction of the image of the two cameras in the end what happened to the essence of the change.

The answer to the corn is: the left and right two graphs are rotated around the optical axis, rotating the two camera's main point line parallel pixel coordinate lines.

The answer to corn is also its own understanding of line alignment, more popular.                    This is different from many of the books in the geometrical terms of the description, we can think about it, Corn said there is no reason. Well, now that it's clear, to reach the line alignment, the image needs to be changed. So let's use mathematical expressions to express them.

Corn drew a simple schematic, painted ugly everyone do not laughed at.

The red line represents the corrected line, which is aligned to the left and right. As you can see, the line from the original image is transferred to the line of the corrected image, the left image turns α, and the right image rotates θ. So how to determine the two angle of rotation.

As you can recall, in the second chapter of this series, we describe the method by which the rotation matrix is decomposed by different axes when the rigid body transforms. Can be seen as rotating along the optical axis, the purpose of rotation is parallel to the main point connection. Let's take the left image as an example: then we set the rotation matrix to R H,

The RH can be expressed as a normalized t (translation vector): The E1 is a rotating pointer, then the E2 is the cross product of the E1 and the optical axis, E3 is the cross product of E2 and E1:

You can further calculate:

We find that the alignment is around the E3 direction, turn α. Similarly, to the right, the figure is to turn θ.

Through the above deduction, we put the three-dimensional calibration and the three-dimensional correction of the mathematical ideas to clarify. In fact, three-dimensional matching is a lot of methods, corn in the only introduction of the basic principle of stereo correction. Other methods, some can not rely on calibration parameters, if you want to study the stereo correction, you can search some classic papers for in-depth study. such as: A. Fusiello, E. Trucco, and A. Verri. Write acompact algorithm for rectification of stereo pairs. Wait a minute.

All preparations for the three-dimensional reconstruction have been completed as of now. There is only one final step left in the geometric frame: stereoscopic imaging. (because this series of posts only describes the geometry of binocular vision, so skip the match) The corn in the next blog post will share with you the final fruit of the geometric context: stereoscopic imaging. This is our ultimate goal for the derivation and understanding of the binocular vision geometry framework.