Camera Calibration (favorites)

Camera calibration is a critical problem in computer vision.

The brightness of each point in the image captured by the camera reflects the intensity of reflected light at a certain point on the surface of the space object, and the position of the point on the image is related to the geometric position of the corresponding point on the surface of the space object, the relationship between these locations is determined by the camera imaging geometric model. The parameters of this geometric model are called camera parameters. These parameters must be determined by the test and calculation. The test and calculation process is called Camera calibration. The camera model simplifies the Geometric relationship of optical imaging. The simplest model is a linear model or a pin-hole model ). The linear model cannot accurately describe the imaging ry of the camera when the camera lens is a wide-angle lens, so a non-linear model is required. Camera calibration is also related to the task of the computer vision system. In stereo vision, two or more cameras are generally used. Therefore, you also need to know the Geometric relationship between cameras. In addition, there are hand-eye calibration and so on. Generally, the camera calibration method requires an object of known shape and size in front of the camera, which is called a calibration object or a reference object ). Because, in some visual systems (such as robot visual systems and active visual systems), the camera's position must be changed frequently or the camera's optical system (such as aperture and focal length) needs to be adjusted, after each adjustment, camera calibration is required. In this case, it is often unrealistic to place a calibration block in the camera's work environment. Camera self-calibration is a calibration method that does not need to be used.

The problems discussed below only involve linear camera calibration.

Linear Camera Calibration involves three coordinate systems: image coordinate system, camera coordinate system, and world coordinate system.

(U, v) indicates the coordinates of the image coordinate system in pixels, and (x, y) indicates the coordinates of the image coordinate system in millimeters. In the x, y coordinate system, the origin o1 is defined at the intersection of the camera's optical axis and the image plane. This point is generally located in the image center. when the camera is created, it may also deviate. If the coordinates of o1 in the u and v coordinate systems are (u0, v0), the physical size of each pixel on the X and Y axes is dx, dy, the coordinates of any pixel in the image in the two coordinate systems are as follows:

The camera imaging relationship can be expressed. The O point is called the camera light center. The XC axis and the YC axis are parallel to the Y axis of the image. The ZC axis is the camera's optical axis, which is perpendicular to the image plane. The intersection of the optical axis and the image plane is the origin of the image coordinate system. The Cartesian coordinate system composed of the O, XC, YC, and ZC axes is called the camera coordinate system. OO1 indicates the camera focal length. Because the camera can be installed in any position in the environment, we also select a reference coordinate system in the environment to describe the camera's position and use it to describe the location of any object in the environment. This coordinate system is called the world coordinate system. It consists of XW, YW, and ZW.

In the linear camera model (pinhole model), the position of any point P in the space in the image can be expressed in the pinhole model, that is, the projection position P of any point P on the image, it is the intersection of the OP and the image plane connecting the optical center O and the P point. This relationship is also called the central projection or perspective projection.

The relationship between the camera coordinate system and the world coordinate system can be described using the rotation matrix R and the translation vector T. Therefore, the following formula is true:

After the transformation of several coordinate systems, a relational relation can be derived:

The above formula is very important for calibration. Among them, Ax = f/dx, ay = f/dy; M is a 3 × 4 matrix, called a projection matrix; M1 is completely determined by ax, ay, U0, and v0, because, ax, ay, U0, and V0 are only related to the internal structure of the camera. We call these parameters the internal parameters of the camera. m2 is completely determined by the orientation of the camera relative to the world coordinate system. It is called the external parameters of the camera, determine the internal and external parameters of a camera, which is called Camera calibration.

Camera Calibration generally requires a special calibration parameter placed in front of the camera. The camera obtains the image of the object and then calculates the internal and external parameters of the camera, the location of each feature point on the reference object relative to the world coordinate system should be accurately determined during creation. The location of the world coordinate system can be selected at any time. Of course, future and present convenience issues must be taken into account. After the projection of these known points on the image is obtained, the internal and external parameters of the camera can be calculated. The specific method is as follows.

The following formula can be derived from this equation.

The formula above indicates that if the calibration block has n known points and their spatial coordinates and image coordinates are known, we will have 2N linear equations about M matrix elements, the following are the equations written in matrix form:

As can be seen from the above formula, multiplying the M matrix by any constant not 0 does not affect the relationship between (XW, YW, ZW) and (u, v). Therefore, you can specify m34 = 1, in this way, we can obtain 2N linear equations for other elements in the M matrix. The number of these unknown elements is 11 and is recorded as the 11-dimensional vector M. Therefore, the formula above can be abbreviated as Km = u, K is a 2n × 11 matrix, M is an unknown 11-dimensional vector, and U is a 2n-dimensional vector. K and u are known vectors. When 2n> 11, use the least square method to obtain the solution of the above linear equation: M = (KTK)-1 ktu

We can find the M matrix from the coordinates of more than six known points in space and their image points. In general calibration, we make dozens of known points on the calibration block, so that the number of equations greatly exceeds the number of unknown numbers, then, the least square method is used to solve the problem to reduce the impact of the error. After finding the M matrix, we can use some derivation to calculate all the internal and external parameters of the camera.

In summary, the M matrix can be obtained from more than six known points in space and their image coordinate, and all the internal and external parameters can be obtained through a certain formula.

Note that the M matrix determines the relationship between the spatial coordinate and its image coordinate. In many applications (such as stereoscopic vision), after calculating the M matrix, it is not necessary to obtain the internal and external parameters of the camera. That is to say, the M matrix itself also represents the camera parameters, but these parameters do not have specific physical meanings. In some documents, they are called hidden parameters. In some applications (such as motion analysis), the M matrix needs to be decomposed to obtain the internal and external parameters of the camera. However, decomposition of internal and external parameters of the camera may cause errors.