Distortion correction of "OPENCV" fisheye image--Calibration correction

Reference:
http://docs.opencv.org/3.0.0/db/d58/group__calib3d__fisheye.html# Gga91b6a47d784dd47ea2c76ef656d7c3dca0899eaa2f96d6eed9927c4b4f4464e05
Http://docs.opencv.org/master/modules/calib3d/doc/calib3d.html
opencv2.4.9 fisheye Camera Model Reference
Kannala J, Brandt s S. A generic camera model and calibration method for conventional, wide-angle, and Fish-eye lenses[j]. Pattern analysis and Machine Intelligence, IEEE transactions on, 2006, 28 (8): 1335-1340.

Fish Eye Lens model
The internal model of the fisheye lens can be expressed as the internal reference of the normal lens, but the distortion parameters are different, meaning the following:

Set (x, Y, z) as a three-dimensional coordinate point, projected on the image of the two-dimensional coordinates (U,V), if the distortion is not considered, the projection relationship is as follows:
　　
R and T represent the rotation matrix and shift vectors in the camera's external parameters respectively.

　　
Calibration Process
First Call OpenCV's Findchessboardcorners () to find the image of the calibration plate corner point, and then according to the size of the calibration plate to specify the three-dimensional coordinates of these corner points, and then call Fisheye::calibrate () to calibrate, Using the internal and distorted parameters of the calibration results, the fisheye::undistortimage () is called to do the image distortion operation. Finally, we call a distorted picture to be tested and use the calibration result to calibrate the distortion.

//Operating environment vs2012+opencv2.4.9#include <opencv2\opencv.hpp>#include <fstream>using namespace STD;using namespaceCvintMain () {Ofstream fout ("Caliberation_result.txt");/** file **/to save the calibration result    /************************************************************************ reads each image, extracts the corners from it, and then pixels the diagonal point to be accurate. *************************************************************************/       cout<<"Start extracting corner points .........."<<endl;intImage_count=4;/**** number of images ****/Size image_size;/**** size of image ****/Size board_size = Size (6,9);/**** the number of corner points per row and column on the calibration board ****/       vector<Point2f>Corners;/**** cache the detected corner points on each image ****/     vector<vector<Point2f>>Corners_seq;/**** Save all detected corner points ****/        vector<Mat>Image_seq;intCount =0; for(inti =0; I! = Image_count; i++) {cout<<"Frame #"<<i+1<<"..."<<endl;stringImagefilename;STD::StringStreamstrstm; strstm<<i+1;        strstm>>imagefilename; Imagefilename + =". jpg"; Cv::mat image = Imread ("img"+imagefilename); Image_size = Image.size ();//image_size = Size (Image.cols, image.rows);        / * Extract Corner points * /Mat Imagegray; Cvtcolor (image, Imagegray, Cv_rgb2gray);BOOLPatternfound = findchessboardcorners (image, Board_size, Corners,calib_cb_adaptive_thresh + CALIB_CB_NORMALIZE_IMAGE+ Calib_cb_fast_check);if(!patternfound) {cout<<"Can not find chessboard corners!\n";Exit(1); }Else{/ * Sub-pixel precision * /Cornersubpix (Imagegray, Corners, Size ( One, One), Size (-1, -1), Termcriteria (Cv_termcrit_eps + cv_termcrit_iter, -,0.1));            Count = Count + corners.size ();        Corners_seq.push_back (corners);    } image_seq.push_back (image); }cout<<"Corner extraction complete! \ n ";/************************************************************************ Camera Calibration ********************* ****************************************************/       cout<<"The beginning of the calibration ... ......"<<endl; Size square_size = Size ( -, -);/**** the size of each checkerboard on the calibration board actually measured ****/       vector<vector<Point3f>>object_points;/**** The three-dimensional coordinate ****/of the upper corner of the calibration board .Mat image_points = Mat (1, Count, CV_32FC2, Scalar::all (0));/***** Save all corner points of the extract *****/        vector<int>point_counts;/***** the number of corner points in each image ****/Mat Intrinsic_matrix = Mat (3,3, CV_32FC1, Scalar::all (0));/***** Camera Internal parameter matrix ****/Mat distortion_coeffs = Mat (1,4, CV_32FC1, Scalar::all (0));/* 4 distortion factor for Camera: K1,K2,P1,P2 */      vector<cv::Mat>Rotation_vectors;/* Rotation vector for each image */       vector<cv::Mat>Translation_vectors;/* Translation vector per image */      / * Initialize the three-dimensional coordinates of the corners on the calibration board * /          for(intt=0; t<image_count;t++) { vector<Point3f>Temppointset; for(intI=0; i<board_size.height;i++) { for(intj=0; j<board_size.width;j++) {/ * Assume that the calibration plate is placed on the z=0 plane in the world coordinate system * /point3f Temppoint;                Temppoint.x = I*square_size.width;                Temppoint.y = J*square_size.height; Temppoint.z =0;            Temppointset.push_back (Temppoint);    }} object_points.push_back (Temppointset); }/ * Initialize the number of corners in each image, here we assume that the complete calibration plate can be seen in each image */        for(intI=0; i< Image_count;       i++) {point_counts.push_back (board_size.width*board_size.height); }/ * Start scaling * /Calibratecamera (object_points, Corners_seq, Image_size, Intrinsic_matrix, Distortion_coeffs, Rotation_vectors, Translation_vectors,0);cout<<"Calibration Complete!" \ n ";/************************************************************************ Evaluation of calibration results ***************** ********************************************************/       cout<<"begin to evaluate the results of the calibration ......."<<endl;DoubleTotal_err =0.0;/ * The sum of the average errors of all images * /       DoubleErr =0.0;/* Average error per image */        vector<Point2f>Image_points2;/**** Save the recalculated projection point ****/       cout<<"Calibration error for each image:"<<endl;cout<<"Calibration error for each image:"<<endl<<endl; for(intI=0;  i<image_count; i++) { vector<Point3f>Temppointset = Object_points[i];/**** through the internal and external parameters of the camera, the three-dimensional point of space is re-projected, and a new projection point is obtained ****/Projectpoints (Temppointset, Rotation_vectors[i], translation_vectors[i], Intrinsic_matrix, distortion_coeffs, image _POINTS2);/* Calculates the error between the new projection point and the old projection point */           vector<Point2f>Tempimagepoint = Corners_seq[i]; Mat Tempimagepointmat = Mat (1, Tempimagepoint.size (), CV_32FC2); Mat Image_points2mat = Mat (1, Image_points2.size (), CV_32FC2); for(size_t i =0; I! = Tempimagepoint.size (); i++) {image_points2mat.at<vec2f> (0, i) = vec2f (image_points2[i].x, IMAGE_POINTS2[I].Y); Tempimagepointmat.at<vec2f> (0, i) = vec2f (tempimagepoint[i].x, TEMPIMAGEPOINT[I].Y);        } err = Norm (Image_points2mat, Tempimagepointmat, NORM_L2); Total_err + = err/= Point_counts[i];cout<<"section"<<i+1<<"Average error of the image:"<<err<<"Pixels"<<endl; fout<<"section"<<i+1<<"Average error of the image:"<<err<<"Pixels"<<endl; }cout<<"Overall average error:"<<total_err/image_count<<"Pixels"<<endl; fout<<"Overall average error:"<<total_err/image_count<<"Pixels"<<endl<<endl;cout<<"The evaluation is complete!" "<<endl;/************************************************************************ Save the calibration result ******************** *****************************************************/       cout<<"Start saving the calibration results .........."<<endl; Mat Rotation_matrix = Mat (3,3, CV_32FC1, Scalar::all (0));/* Save the rotation matrix for each image */fout<<"in-Camera parameter matrix:"<<endl;       fout<<intrinsic_matrix<<endl; fout<<"distortion factor: \ n"; fout<<distortion_coeffs<<endl; for(intI=0; i<image_count; i++) {fout<<"section"<<i+1<<"The rotation vector of the image:"<<endl; fout<<rotation_vectors[i]<<endl;/ * Convert the rotation vector to the corresponding rotation matrix * /Rodrigues (Rotation_vectors[i],rotation_matrix); fout<<"section"<<i+1<<"The rotation matrix of the image:"<<endl;           fout<<rotation_matrix<<endl; fout<<"section"<<i+1<<"The translation vector of the image:"<<endl;       fout<<translation_vectors[i]<<endl; }cout<<"Finish Saving"<<endl; fout<<endl;/************************************************************************ Show Calibration Results ******************** *****************************************************/Mat mapx = Mat (IMAGE_SIZE,CV_32FC1);    Mat mapy = Mat (IMAGE_SIZE,CV_32FC1); Mat R = Mat::eye (3,3, cv_32f);cout<<"Save Corrective Images"<<endl; for(inti =0; I! = Image_count; i++) {cout<<"Frame #"<<i+1<<"..."<<endl; Mat Newcameramatrix = Mat (3,3, Cv_32fc1,scalar::all (0));        Initundistortrectifymap (intrinsic_matrix,distortion_coeffs,r,intrinsic_matrix,image_size,cv_32fc1,mapx,mapy);        Mat t = Image_seq[i].clone (); Cv::remap (IMAGE_SEQ[I],T,MAPX, Mapy, inter_linear);stringImagefilename;STD::StringStreamstrstm; strstm<<i+1;        strstm>>imagefilename; Imagefilename + ="_d.jpg";    Imwrite (imagefilename,t); }cout<<"Save End"<<endl;/************************************************************************ test a picture ******************** *****************************************************/    cout<<"Testimage ..."<<endl; Mat Newcameramatrix = Mat (3,3, Cv_32fc1,scalar::all (0)); Mat testimage = Imread ("Test.jpg",1);    Initundistortrectifymap (intrinsic_matrix,distortion_coeffs,r,intrinsic_matrix,image_size,cv_32fc1,mapx,mapy);    Mat t = Testimage.clone ();    Cv::remap (TESTIMAGE,T,MAPX, Mapy, inter_linear); Imwrite ("Testoutput.jpg", t);cout<<"Save End"<<endl;return 0;}

Experimental Results:
Average error for 1th Image: 0.234066 pixels
Average error for 2nd Image: 0.174135 pixels
Average error for 3rd Image: 0.230404 pixels
Average error for 4th Image: 0.385148 pixels
Overall average error: 0.255938 megapixels

In-camera parameter matrix:
[1526.985633757815, 0, 950.2799160336148;
0, 1524.888027372896, 1626.73459851929;
0, 0, 1]
Distortion factor:
[-0.2939854304014771,-0.03809991415256448,-0.0006251660855326396, 0.002477996377628106, 0.08710997961828348]
Rotation vector for the 1th image:
[-2.044082716917989;
2.112604062430628;
-0.4512837261687678]
Rotation matrix for the 1th image:
[-0.04783928023483552,-0.9443057645861673, 0.3255733807238344;
-0.9949113651213172, 0.01611256042205056,-0.0994573323042737;
0.08867231145656364,-0.3286746238546258,-0.9402713506296841]
Translation vectors for the 1th image:
[64.01021865713052;
98.56104619426411;
147.9806640258283]
Rotation vector for the 2nd image:
[-1.162650885638092;
2.737696507332989;
0.1448079633779595]
Rotation matrix for the 2nd image:
[-0.6837940150923046,-0.7210082406540599, 0.1121287734387665;
-0.7051562408233225, 0.6924689388697722, 0.1524514503981244;
-0.1875644447966232, 0.02517708500816826,-0.9819295766187237]
Translation vectors for the 2nd image:
[92.91837801221574;
58.91364372752962;
181.052631025211]
Rotation vector for the 3rd image:
[1.684131027388537;
-2.175687329421208;
-0.2403921029056955]
Rotation matrix for the 3rd image:
[-0.2115618190438919,-0.894254676073194,-0.3943984927000438;
-0.9587869203326895, 0.2681969799312525,-0.09379776838876375;
0.1896555776184723, 0.3583000896971452,-0.9141399278016584]
Translation vectors for the 3rd image:
[35.54570544850407;
64.77270398447031;
122.9118368955376]
Rotation vector for the 4th image:
[2.717206221016452;
1.219573939678138;
0.807283561760737]
Rotation matrix for the 4th image:
[0.5510728665512401, 0.6808924688738628, 0.4823941765629592;
0.7100594861195663,-0.6862919236796685, 0.1575402223677423;
0.4383311783601724, 0.2557124191800546,-0.8616710142243772]
Translation vectors for the 4th image:
[-45.13197759830062;
-19.42296234432086;
113.2028514971537]

Distorted Image:
Img1.jpg

Img2.jpg

Img3.jpg

Img4.jpg

Test.jpg

correction Result:




Testoutput.jpg

Code Download:
http://download.csdn.net/detail/qq_15947787/9523484

Distortion correction of "OPENCV" fisheye image--Calibration correction