OPENCV Camera Calibration

Camera Calibration

Camera calibration: Simply speaking, it is the process of acquiring camera parameters. Parameters such as: in-camera parameter matrix, projection matrix, rotation matrix and pan-shift matrix, etc.

• What is camera parameters?

Simply put, the real world of people, objects, shot into a picture (two-dimensional). The three-dimensional coordinates of a person or thing in the world, and the relationship between two-dimensional coordinates in the image. What is the representation of the relationship between two different dimensional coordinates? With camera parameters.

• Imaging principle of the camera

First look at the camera's imaging principle:

, this is a camera model. The simplification of objects is regarded as a point. The light from the object, through the lens, hits the image plane (image sensor), which is imaged. D0 is the distance from the object to the lens, the distance from the lens to the image plane, and F is the focal length of the lens. The following relationships are met by the three.

now, simplify the camera model above.
The aperture of the camera is considered to be infinitely small, and only the radial of the center is taken into account, which ignores the influence of the lens. Then, because the D0 is much larger than Di, the image plane is placed at the focal length so that the object is imaged on the image plane as an inverted image (without the influence of the lens, only the light entering from the center aperture is considered). This simplified model is the pinhole camera model. Then, in front of the camera, we put the image plane in the distance of the focal length, we can easily get a straight image (not inverted). Of course, this is only theoretical, you can't take the image sensor out of the camera and put it in front of the lens. In practice, the pinhole camera should be the inverted image after imaging to obtain a positive image.
In this case, we obtained a simplified model, such as:

H0 is the height of the object, HI is the height of the object on the image, F is the focal length (distance), D0 is the distance from the image to the lens. The four meet the following relationships:
(1)
The height of the object in the image is inversely proportional to the d0. That is, the farther away from the lens, the smaller the object in the image, the closer it gets (well, this is nonsense).
But with this formula, we can predict the position of objects in three dimensions in the image (two-dimensional). So how do you predict?

• Camera Calibration

As shown, consider a point in the three-dimensional world, and its coordinate relationship in the image (two dimensions), based on the simplified model above.

(x, y, z) is the three-dimensional coordinate of a point, which is the coordinates of the image (two-dimensional) after it is imaged by the camera. U0 and V0 are the center point of the camera (the main point), which is located in the center of the image plane (theoretically so.) But the actual camera will have a few pixel deviations)
Now consider only the y direction, because the need to convert the coordinates in the three-dimensional world to the pixels on the image (the coordinates on the image, actually the position of the pixels), need to ask the focal length in the y direction equal to how many pixels (with pixel values to indicate the focal length), PY is the high pixel, the focal length F (m or The vertical pixels represent a focal length of
According to the formula (1), only the Y direction is considered. We have a point in the three-dimensional world, the coordinates of y in the image (two dimensions).

In the same vein, the coordinates of X are obtained.

Now, move the Origin o in the coordinate system to the upper-left corner of the image. Because (x, y) is an offset to (u0,v0), the above indicates that the coordinates of the midpoint of the image (two dimensions) are unchanged. Rewrite the equation in the form of a matrix.

Where the first matrix to the left of the equation is called the "in-Camera parameter matrix" and the second matrix is called (projection matrix).

In a more general case, the reference coordinate system at the beginning is not at the main point (the center point) and requires an additional two parameters "rotation vector" and "translation vector" to represent the equation, which are different from one another. After the integration, the above formula is rewritten as.

• Correction Distortion
  After the camera calibration is obtained, the camera parameters can be calculated by calculating two mapping functions (x and Y coordinates), which give the image coordinates without distortion respectively. The distorted image is re-mapped into an image without distortion.

Code:

When doing camera calibration, a set of images is usually taken with a calibration board (checkerboard), which uses these images to extract corner points, and calculates the camera parameters through the coordinates of the corner points in the image and coordinates in the three-dimensional world (usually custom 3-D coordinates).

std::vector<cv::Point2f>imageConers;//提取标定图像角点，保存角点坐标（二维） cv::findChessboardCorners(image,  //角点数目如（6,4）六行，四列  imageConers);

The function Calibratecamera completes the camera calibration work.

cv::calibrateCamera(objectPoints,//三维坐标//二维坐标 imageSize,//图像大小 camerMatirx,//相机内参数矩阵 disCoeffs,//投影矩阵  //旋转  tvecs,//平移flag  //标记opencv提供几种参数，可以参看在线的opencv document);

Calculation of distortion parameters, de-distortion

//计算畸变参数cv::initUndistortRectifyMap(camerMatirx, disCoeffs,    cv::Mat(), cv::Mat(), image.size(), CV_32FC1,     //x映射函数    map2  //y映射函数    );//应用映射函数cv::remap//畸变图像//去畸变图像map1, map2, cv::INTER_LINEAR);

Now consolidate the code.

• Example:

header. h

#include <opencv2\core\core.hpp>#include <opencv2\highgui\highgui.hpp>#include <opencv2\imgproc\imgproc.hpp>#include <opencv2\calib3d\calib3d.hpp>#include <opencv2/features2d/features2d.hpp>#include <string>#include <vector>classcameracalibrator{Private://World coordinates    STD:: Vector < std::vector<CV::P oint3f >>objectpoints;//Image coordinates    STD:: Vector <std::vector<CV::P oint2f>>imagepoints;//Output matrixCv::mat Camermatirx; Cv::mat discoeffs;//Mark    intFlag//de-distortion parametersCv::mat Map1, MAP2;//whether to distort    BOOLMustinitundistort;/// save point data    voidAddpoints (Const STD:: vector<CV::P oint2f>&imageconers,Const STD:: vector<CV::P oint3f>&objectconers) {imagepoints.push_back (imageconers);    Objectpoints.push_back (objectconers); } Public: Cameracalibrator (): Flag (0), Mustinitundistort (true){}//Open board picture, extract corner point    intAddchessboardpoints (Const STD:: Vector<std::string>&filelist,cv::size &boardsize) {STD:: vector<CV::P oint2f>Imageconers;STD:: vector<CV::P oint3f>Objectconers;//Enter the world coordinates of the corner point         for(inti =0; i < boardsize.height; i++) { for(intj =0; J < Boardsize.width; J + +) {objectconers.push_back (CV::P oint3f (i, J,0.0f)); }        }//Calculate the coordinates of the corner in the imageCv::mat image;intSuccess =0; for(inti =0; I < filelist.size (); i++) {image = Cv::imread (Filelist[i],0);//Find corner coordinates            BOOLFound = cv::findchessboardcorners (image, Boardsize, imageconers); Cv::cornersubpix (image, Imageconers, Cv::size (5,5), Cv::size (-1, -1), Cv::termcriteria (Cv::termcriteria::max_iter + cv::termcriteria::eps, -,0.1));if(imageconers.size () = = Boardsize.area ())                {addpoints (imageconers, objectconers);            success++; }//Draw out corner pointsCV::d rawchessboardcorners (Image, Boardsize, imageconers, found); Cv::imshow ("Corners on chessboard", image); Cv::waitkey ( -); }returnSuccess }//Camera Calibration    DoubleCalibrate (cv::size&imagesize) {Mustinitundistort =true;STD:: vector<cv::Mat>Rvecs, Tvecs;//Camera Calibration        returnCv::calibratecamera (objectpoints, imagepoints, ImageSize, Camermatirx, Discoeffs, Rvecs, tvecs, flag); }// to distortionCv::mat Remap (ConstCv::mat &image) {Cv::mat undistorted;if(Mustinitundistort) {//calculation of distortion parametersCv::initundistortrectifymap (Camermatirx, Discoeffs, Cv::mat (), Cv::mat (), Image.size (), CV_32FC1, MAP1, map            2); Mustinitundistort =false; }//Apply mapping functionCv::remap (image, undistorted, Map1, MAP2, cv::inter_linear);returnundistorted; }///constant member function to obtain the internal parameter matrix and projection matrix data of the cameraCv::mat Getcameramatrix ()Const{returnCamermatirx; } Cv::mat getdistcoeffs ()Const{returndiscoeffs; }};

Source. cpp

#include "header. h"#include <iomanip>#include <iostream>intMain () {Cameracalibrator Cc; Cv::mat image;STD:: Vector<std::string>FileList Cv::namedwindow ("Image"); for(inti =1; I <= A; i++) {/// Read pictures        STD::StringStreamS S <<"D:/images/chessboards/chessboard"<<STD:: SETW (2) <<STD:: Setfill (' 0 ') << I <<". jpg";STD::cout<< s.str () <<STD:: Endl;        Filelist.push_back (S.str ()); Image = Cv::imread (S.str (),0); Cv::imshow ("Image", image); Cv::waitkey ( -); }//Camera CalibrationCv::size Boardsize (6,4);    Cc.addchessboardpoints (filelist, boardsize); Cc.calibrate (Image.size ());//de-distortionImage = Cv::imread (filelist[1]);    Cv::mat Uimage=cc.remap (image); Cv::imshow ("Original image", image); Cv::imshow ("to distort", uimage);//Display in-Camera parameter matrixCv::mat Cameramatrix = Cc.getcameramatrix ();STD::cout<<"Camera intrinsic:"<< cameramatrix.rows <<"x"<< Cameramatrix.cols <<STD:: Endl;STD::cout<< cameramatrix.at<Double> (0,0) <<" "<< cameramatrix.at<Double> (0,1) <<" "<< cameramatrix.at<Double> (0,2) <<STD:: Endl;STD::cout<< cameramatrix.at<Double> (1,0) <<" "<< cameramatrix.at<Double> (1,1) <<" "<< cameramatrix.at<Double> (1,2) <<STD:: Endl;STD::cout<< cameramatrix.at<Double> (2,0) <<" "<< cameramatrix.at<Double> (2,1) <<" "<< cameramatrix.at<Double> (2,2) <<STD:: Endl; Cv::waitkey (0);}

Experimental results:


To see that the parameter matrix inside the camera is
172.654, 0, 157.829
0, 184.195, 118.635
0, 0, 1

OpenCV Camera Calibration