[OpenCV] ry distortion correction and local enhancement, opencv ry distortion correction

[OpenCV] ry distortion correction and local enhancement, opencv ry distortion correction
1. Principle

During image acquisition, due to the non-linearity of the imaging system and the changes in the attitude of the aircraft, the image after imaging may be out of proportion or even distorted compared with the original image. This type of image degradation is known as geometric distortion (distortion ).
This is due to the non-linearity of the imaging system, the angle of view changes during video recording, and the bending of the surface of the subject. For example, because the scanning and deflection systems of the video camera and the cathode ray tube display are non-linear, the pillow or bucket distortion is often used, and the image perspective distortion obtained from the strabismus angle is often used.
Geometric distortion is mainly caused by the displacement of pixels in the image. It is typically manifested in the distortion of objects in the image and the imbalance between distance and distance. The solution to this type of distortion problem is geometric distortion correction (collectively referred to as geometric correction.
There are two kinds of Correction Methods for geometric distortion caused by the imaging system: one is the pre-distortion method, that is, the non-linear scanning Deflection Method opposite to the distortion is used to offset the expected image distortion; another method is the posterior correction method, which uses polynomial curves to fit each distorted network cable horizontally and vertically, and then calculates the inverse transformation correction function, use this correction function to correct the distorted image.
Geometric distortion correction is divided into two steps: the first step is to perform geometric transformation on the coordinate space of the original image so that the pixel falls in the correct position; the second step is to re-determine the gray value of the new pixel, this is because after the above coordinate transformation, some pixels are sometimes squeezed together and sometimes dispersed, so that the corrected pixels do not fall on discrete coordinate points, therefore, you need to re-determine the gray value of these pixels.

• Description of Geometric Distortion

Any geometric level can be defined by the non-distortion coordinate system (x, y) transformation to the distortion coordinate system (x ', y') equation.
If f (x, y) is the original image without distortion, g (x', y') is the result of f (x, y) distortion, this distortion process is known and can be defined by functions h1 (x, y) and h2 (x, y), so there are:
This is the basic relationship of geometric correction. The restoration of such distortion is actually a ing transformation problem.

• Geometric Transformation

From the basic relationship of geometric correction, we can see that the original image f (x, y) is required when the distorted image g (x', y') is known) the key is the required functions h1 (x, y) and h2 (x, y). Then the f (x, y) method is simpler. However, in practice, h1 (x, y) and h2 (x, y) are usually unknown. In this case, we can adopt posterior correction.
Generally, h1 (x, y), h2 (x, y) can be used to approximate the number of times where N is a polynomial, and aij and bij are the undetermined coefficients of their respective terms.
The principle of posterior correction is to fit the coefficients of the above two polynomials through the correspondence between known correct pixel points and distortion points, the fitted polynomials serve as the basis for transformation to restore other distortion points. For example, a reference image uses an imaging system to form a distorted image. By studying the correspondence between the reference image and the distorted image, we can find the coefficients of polynomials.
When N = 1, the transformation is linear. This linear distortion can be used to approximate a small geometric distortion. However, due to the complexity and diversity of the actual situation, the above formula N = 2 or more may not be able to find the optimal solution. At this time, we need to use the least square method.

• Local Enhancement

The local enhancement method calculates the transform or filter parameters based on the characteristics of the local area of interest, and applies the results to the local area of interest to obtain the desired enhancement effect. It can be seen that, compared with the global enhancement method, the local enhancement method has an additional step before the specific enhancement operation, the enhancement methods described in the previous sections can be used for each local area.
Histogram transformation is the most commonly used method for Spatial enhancement. It is also easy to be used for local image enhancement. You only need to first divide the image into a series of (generally overlapping) Small Areas (sub-images). At this time, histogram equalization or normalization can be performed based on the Pixel Distribution in the residential area, in this way, each small area has different enhancement effects.

 

 

2. Method

Use interpolation to determine the gray value of a pixel. After the coordinate (x, y) of the original image is transformed, it falls into the distorted image, but it is not just on the image pixel. You need to obtain the gray value through some means, common methods include the nearest neighbor method, bilinear interpolation method, and cubic convolution method.

• Nearest Neighbor Method

A simple interpolation method is the nearest neighbor method, and the grayscale value of the input pixel closest to the position mapped to it is the difference result. If the pixels whose coordinates are (x, y) on the original image are transformed and fall into the distorted image g (this interpolation method greatly changes the gray value of neighboring pixels, but the subtle structure is rough.

• Bilinear interpolation

A pixel coordinate on the original image f (x, y) is (x, y), after transformation, falls into the distorted image g (x', y ') the coordinates are (u, v), and [] in the following formula indicates the integer.
Definition: a = u-[u], B = v-[v], then the value of g (u, v) is calculated according to the following formula:
When u = [u] Or v = [v ],
Restore the image. This is the bilinear interpolation method.
Compared with the nearest neighbor method, the inner interpolation method performs gray-scale continuous correction, and the results generally meet the requirements. However, the computation is large and low-pass, and the image contour is blurred. To further improve image quality, you can use cubic convolution.

• Local Enhancement

Local enhancement can also be used directly to enhance the entire image to achieve different local enhancements. For example, a common method for Local airspace enhancement is to use the mean and variance of pixels in the neighborhood of each pixel. Here, the mean is an average brightness measure, while the variance is a contrast measure. Specifically, to increase the input graph f (x, y) to the output graph g (x, y), perform the following transformation at each pixel position (x, y:
It is called a local gain function.
In the above two formula, m (x, y) is the gray mean in the neighborhood centered on pixels (x, y); σ (x, y) is the pixel (x, y) the gray mean squared difference in the center neighborhood, M is the average gray scale value of f (x, y), and k is a proportional constant.
The difference between A (x, y) and f (x, y) and m (x, y) can enlarge the local variation of the image, because A (x, y) it is inversely proportional to the mean variance, so the gain obtained in the area with a smaller contrast in the image is larger, so that local enhancement can be achieved.
In formula (4.7.1), m (x, y) is added back to restore the average gray value of the original region. In reality, in order to balance the offset of the gray value of the isolated area in the image, only A part of m (x, y) is often added back, and A (x, y) is usually) within a certain range.

 

 

3. key code

• Nearest Neighbor interpolation (Forward ing)

Int main () {IplImage * img = cvLoadImage ("Image/slant.jpg", 0); IplImage * out = cvCreateImage (cvGetSize (img), img-> depth, img-> nChannels); // manually specify the Quadrilateral vertices. A, B, C, and D are pre-mapped (skewed), and Ao, Bo, Co, and Do are mapped (corrected) cvPoint A = cvPoint (475,191); CvPoint B = cvPoint (355,399); CvPoint C = cvPoint (25,209); CvPoint D = cvPoint (); CvPoint Ao = cvPoint ); cvPoint Bo = cvPoint (Hangzhou, 80); CvPoint Co = cvPoint (440,320); CvPoint Do = cvPoint (60,320); float m [16] = {. x,. y,. x *. y, 1, B. x, B. y, B. x * B. y, 1, C. x, C. y, C. x * C. y, 1, D. x, D. y, D. x * D. y, 1}; float n1 [4] = {Ao. x, Bo. x, Co. x, Do. x}; float n2 [4] = {Ao. y, Bo. y, Co. y, Do. y}; CvMat * M = cvCreateMat (4,4, CV_32FC1); CvMat * N1 = cvCreateMat (4,1, CV_32FC1); CvMat * N2 = cvCreateMat (4,1, CV_32FC1 ); cvMat * K4 = cvCreateMat (4,1, CV_32FC1); CvMat * K8 = cvCreateMat (4,1, CV_32FC1); cvSetData (M, m, CV_AUTOSTEP); cvSetData (N1, n1, CV_AUTOSTEP ); cvSetData (N2, n2, CV_AUTOSTEP); // solves the forward ing equations and obtains eight parameters cvSolve (M, N1, K4, CV_LU); cvSolve (M, N2, K8, CV_LU); float k1 = K4-> data. fl [0]; float k2 = K4-> data. fl [1]; float k3 = K4-> data. fl [2]; float k4 = K4-> data. fl [3]; float k5 = K8-> data. fl [0]; float k6 = K8-> data. fl [1]; float k7 = K8-> data. fl [2]; float k8 = K8-> data. fl [3]; CvScalar s; // source // forward ing for (int x = 0; x  width; x ++) {for (int y = 0; y  height; y ++) {// nearest neighbor int I = int (k1 * x + k2 * y + k3 * x * y + k4 + 0.5 ); int j = int (k5 * x + k6 * y + k7 * x * y + k8 + 0.5 ); if (I <0 | I> = out-> width | j <0 | j> = out-> height) {// discard the part that exceeds the edge after rotation, white space filled by default} else {s = cvGet2D (img, y, x); cvSet2D (out, j, I, s) ;}} cvNamedWindow ("Source ", CV_WINDOW_AUTOSIZE); cvNamedWindow ("Front", CV_WINDOW_AUTOSIZE); cvShowImage ("Source", img); cvShowImage ("Front", out); cvWaitKey (0 ); cvReleaseImage (& img); cvReleaseImage (& out); cvDestroyWindow ("Source"); cvDestroyWindow ("Front ");}

• Nearest Neighbor interpolation (backward ing)

Difference from forward ing: // manually specify A quadrilateral vertex. A, B, C, and D are before ing (Correction ), after Ao, Bo, Co, and Do are mapped (skewed), CvPoint A = cvPoint (60, 80), CvPoint B = cvPoint (440,320), and CvPoint C = cvPoint ); cvPoint D = cvPoint (60,320); CvPoint Ao = cvPoint (475,191); CvPoint Bo = cvPoint (355,399); CvPoint Co = cvPoint (25,209); CvPoint Do = cvPoint ); ...... // Backward ing for (int x = 0; x <out-> width; x ++) {for (int y = 0; y <out-> height; y ++) {// nearest neighbor int I = int (k1 * x + k2 * y + k3 * x * y + k4 + 0.5 ); int j = int (k5 * x + k6 * y + k7 * x * y + k8 + 0.5 ); if (I <0 | I> = img-> width | j <0 | j> = img-> height) {// fill the outer part of the edge with white by default before rotation} else {s = cvGet2D (img, j, I); cvSet2D (out, y, x, s );}}}......

• Bilinear interpolation (backward ing)

Difference from nearest neighbor interpolation :...... CvScalar s, a, B, c, d; // a, B, c, and d are the four nearest neighbor pixels of s. // backward ing for (int x = 0; x <out-> width; x ++) {for (int y = 0; y <out-> height; y ++) {// bilinear float xx = k1 * x + k2 * y + k3 * x * y + k4; float yy = k5 * x + k6 * y + k7 * x * y + k8; int I = int (xx); int j = int (yy ); if (I <0 | I> = img-> width-1 | j <0 | j> = img-> height-1) {// by default, pixels on the boundary are filled in white outside the edge before rotation, but not processed here} else {a = cvGet2D (img, j, I ); B = cvGet2D (img, j, I + 1); c = cvGet2D (img, j + 1, I); d = cvGet2D (img, j + 1, I + 1); float e = (xx-I) * (B. val [0]-. val [0]) +. val [0]; float f = (xx-I) * (d. val [0]-c. val [0]) + c. val [0]; // calculate the inserted gray value int value = int (yy-j) * (f-e) + e + 0.5); s = cvScalar (value, 0, 0); cvSet2D (out, y, x, s );}}}......

• Local Enhancement

IplImage * img = cvLoadImage ("Image/leaf.jpg", 0); IplImage * out = cvCreateImage (cvGetSize (img), img-> depth, img-> nChannels ); float k = 0.5; // enhancement coefficient CvScalar avg = cvAvg (img); float M = avg. val [0]; // calculate the average gray scale value M std: pair <float, float> cal; for (int I = 1; I  width-1; I ++) {for (int j = 1; j  height-1; j ++) {cal = Calculate (img, I, j ); // custom method int cur = cvGet2D (img, j, I ). val [0]; int enh; if (cal. second = 0) // Prevent the divisor from being 0. If not processed, the result is 0 when the gray scale after enhancement is calculated, and the image is black. Enh = cur; else // use the local enhancement function to calculate the gray value and rounding it into enh = k * (M/cal. second) * (cur-cal. first) + cal. first + 0.5; // gray value cropping to avoid invalid gray value enh = enh <0? 0: enh; enh = enh> 255? 255: enh; cvSet2D (out, j, I, cvScalar (enh ));}}...... // For the custom method, Calculate the mean and standard deviation of the 3*3 neighbor std: pair <float, float> Calculate (IplImage * img, int I, int j) {int sum = 0; int data [9]; int n = 0; for (int a =-1; a <2; a ++) {for (int B =-1; B <2; B ++) {data [n] = cvGet2D (img, j + B, I + ). val [0]; sum + = data [n]; n ++ ;}} float avg = sum/9.0; float tmp = 0; for (n = 0; n <9; n ++) tmp + = pow (data [n]-avg, 2); float sd = sqrt (tmp/9.0); return std :: make_pair (avg, sd );}

 

 

4. Results

After the OpenCV and Visual Studio2012 environments are configured, run the above Code. The output result is shown in.(Reduced the image size for typographical purposes. Please zoom in and observe properly to improve the effect.)
Figure 1 Original Image
 
Figure 2 nearest neighbor interpolation
(Forward ing, with gaps and obvious particles)
 
Figure 3 nearest neighbor interpolation
(Backward ing, with no gaps but obvious particles)
 
Figure 4 bilinear interpolation
(Backward ing, no gaps and the transition is relatively natural)
 
Figure 5 source Image
(Ye Mai is vague and the details are not clear)
 
After Figure 6 is locally enhanced
(Enhancement coefficient k = 0.5, clearer texture)