Opencv2: geometric transformation of images, translation, mirroring, scaling, and rotation (2)

In opencv2: geometric transformation of images, translation, mirroring, scaling, and rotation (1) this paper mainly introduces the forward ing, backward ing in image transformation, and the Interpolation Algorithm for Floating Point Coordinate pixel values in the process of transformation. Based on opencv2, two simple geometric transformations are implemented: Translation and image transformation. This article mainly involves two geometric transformations of slightly complex points: Scaling and rotation.

1. Image Scaling

Image Scaling is mainly used to change the image size. after scaling, the width and height of the image change. The horizontal scaling coefficient controls the scaling of the image width. If the value is 1, the image width remains unchanged. The vertical scaling coefficient controls the scaling of the image height. The value is 1, the Image Height remains unchanged. If the horizontal scaling coefficient is not the same as the vertical scaling coefficient, the ratio of the width and height of the scaled image changes, causing deformation of the image. To keep the ratio of the image width and height unchanged, the horizontal and vertical scaling coefficients must be equal.

The horizontal scaling coefficient and Vertical Scaling coefficient of the left image are both 0.5; the horizontal scaling coefficient of the right image is 1, and the vertical scaling coefficient is 0.5. after scaling, the image width and height ratio change, image deformation.

1.1 Scaling Principle

Set the horizontal scaling coefficient to Sx, the vertical scaling coefficient to Sy, (x0, y0) to the pre-scaling coordinate, and (x, y) to the scaled coordinate ing relationship:

Matrix Representation:

This is a forward ing. In the scaling process, the image size is changed. Using forward ing will cause the problem of overlapping ing and incomplete ing. So here we are more concerned with backward ing, that is, the output image uses the backward ing relationship to find the corresponding pixels in the original image.

Backward ing:

1.2 opencv-Based Scaling implementation

When scaling an image, you must first calculate the size of the scaled image. Set newwidth and newheight to the width and height of the scaled image. width and height are the width and height of the original image, there are:

Then, traverse the scaled image and calculate the scaled pixel position in the original image based on the backward ing relationship. If the obtained floating point coordinate is used, the interpolation algorithm is used to obtain the approximate pixel value.

According to the formula above, the scaled image width and height are multiplied by the original image width and height by the scaling factor.

int rows = static_cast<int>(src.rows * xRatio + 0.5);int cols = static_cast<int>(src.cols * yRatio + 0.5);

Floating Point Coordinates may be obtained during backward ing. The nearest neighbor interpolation and bilinear interpolation are used for processing.

Nearest Interpolation

for (int i = 0; i < rows; i++){            int row = static_cast<int>(i / xRatio + 0.5);            if (row >= src.rows)                row--;            origin = src.ptr<uchar>(row);            p = dst.ptr<uchar>(i);            for (int j = 0; j < cols; j++){                int col = static_cast<int>(j / yRatio + 0.5);                if (col >= src.cols)                    col--;                p[j] = origin[col];            }        }

The nearest interpolation only needs to perform the "Rounding" Operation on the floating point coordinates. However, when rounding the image, the result may exceed the boundary of the original image (only 1 larger than the boundary). Therefore, the following correction is required.

Bilinear interpolation

The accuracy of bilinear interpolation is much better than that of the nearest interpolation, and the calculation amount is much greater. Bilinear interpolation uses the values of Four pixels around the floating point coordinates to mix and obtain the pixel values of the floating point coordinates according to a certain proportion.

Let's set floating point coordinate F. The four integer coordinates around it are T1, T2, T3, and T4, and the absolute value of the vertical axis difference between F and the integer coordinate in the upper left corner is n, the absolute value of the horizontal axis difference is N. According to the analysis in the previous article, the pixel value T of the floating point coordinate F is obtained. The formula is as follows:

F1 is ([F. Y], F. X), F2 is ([F. Y] + 1, F. X ). For details, see opencv2: geometric transformation, translation, image, scaling, and rotation of images (1 ).

 

In implementation, the four integer coordinates around the floating point must be calculated based on the floating point coordinates.

double row = i / xRatio;            double col = j / yRatio;            int lRow = static_cast<int>(row);            int nRow = lRow + 1;            int lCol = static_cast<int>(col);            int rCol = lCol + 1;            double u = row - lRow;            double v = col - lCol;

The coordinates (I, j) of the scaled and scaled image are mapped to the original image (I/xratio, J/yratio ), then find the four integer coordinates (LCOL, lrow), (LCOL, nrow ),

(Rcol, lrow), (rco1, nrow ). The following uses the bilinear interpolation formula to obtain the pixel value of the floating point coordinate.

// If (row> = SRC. rows-1) & (COL> = SRC. cols-1) {lastrow = SRC. PTR <vec3b> (lrow); P [J] = lastrow [LCOL];} // The last line of else if (row> = SRC. rows-1) {lastrow = SRC. PTR <vec3b> (lrow); P [J] = V * lastrow [LCOL] + (1-v) * lastrow [rcol];} // The Last else if (COL> = SRC. cols-1) {lastrow = SRC. PTR <vec3b> (lrow); nextrow = SRC. PTR <vec3b> (nrow); P [J] = u * lastrow [LCOL] + (1-u) * nextrow [LCOL];} else {lastrow = SRC. PTR <vec3b> (lrow); nextrow = SRC. PTR <vec3b> (nrow); vec3b F1 = V * lastrow [LCOL] + (1-v) * lastrow [rcol]; vec3b F2 = V * nextrow [LCOL] + (1-v) * lastrow [rcol]; P [J] = u * F1 + (1-u) * F2 ;}

Because we use four pixels for calculation, there will be non-existent pixels at the boundary. Here we will process three special cases in the bottom right corner, last row, and last column of the image separately.

2. Image Rotation 2.1 rotation principle

The rotation of an image is to let the image rotate the specified angle according to a certain point. The image will not be deformed after rotation, but its vertical symmetric pumping and horizontal symmetric axes will change. After rotation, the relationship between the image coordinates and the original image coordinates cannot be obtained through simple addition and subtraction multiplication, however, a series of complex operations are required. The width and height of the image change after rotation, and the coordinate origin of the image changes.

The coordinate system used by the image is not a common Cartesian system. The upper left corner of the system is its coordinate origin. The X axis is right along the horizontal direction, and the Y axis is down along the vertical direction. In the process of rotation, the Cartesian coordinate system with the center of rotation as the coordinate origin is generally used. Therefore, the first step of image rotation is the transformation of the coordinate system. If the center of rotation is (x0, y0), (x', y') is the coordinate after rotation, and (x, y) is the coordinate after rotation, the coordinate transformation is as follows:

Matrix Representation:

In the final implementation, it is often used to find the corresponding position of the scaled image in the original image through the ing relationship. This requires the inverse transformation of the above ing.

After the coordinate system is transformed to the origin of the rotation center, the coordinates of the image are transformed.

Rotate the coordinates (x0, y0) in a clockwise direction to obtain (x1, Y1 ).

Before rotation:

After rotating:

Matrix Representation:

Its Inverse Transformation:

Since the rotation is centered on the coordinate origin, after the rotation ends, you need to move the coordinate origin to the upper left corner of the image, that is, perform a transformation. Note that the coordinates of the rotation center (x0, y0) are obtained from the coordinate system with the coordinate origin in the upper left corner of the original image, after rotation, the width and height of the image change, which leads to the coordinate origin of the image after rotation and the transformation before rotation.

The above two figures clearly show that the upper left corner of the image before and after rotation, that is, the coordinate origin, is changed.

Before finding the coordinates in the upper left corner of the image after rotation, let's take a look at the width and height of the image after rotation. It can be seen that the width and height of the rotated image are related to the position after the four corners of the original image are rotated.

Set top to the ordinate value of the highest point after rotation, down to the ordinate value of the lowest point after rotation, left to the leftmost point after rotation, and right to the abscissa of the rightmost point after rotation.

If the width and height after rotation are newwidth and newheight, the following relationship is obtained:

It is easy to get the coordinates (left, top) in the upper left corner of the rotated image (Coordinate System with the center of rotation as the origin)

After the rotation is complete, the coordinate system must be converted to the coordinate origin point in the upper left corner of the image. The following transformation relationship can be obtained:

Matrix Representation:

Its Inverse Transformation:

Based on the above, that is to say, the pixel coordinates of the original image must undergo three coordinate transformations:

1. Converts the coordinate origin point from the upper left corner of the image to the center of rotation.
2. Take the center of rotation as the origin, image rotation angle
3. After the rotation ends, the coordinate origin is transformedUpper left corner of the rotated image

The following rotation formula is obtained: (x', y') coordinates after rotation, (x, y) original coordinates, (x0, y0) rotation center, and a rotation angle (clockwise)

In this way, the coordinates of the output image are mapped forward by the input image through ing. The commonly used backward ing is its inverse operation.

2.2 opencv-based implementation

After obtaining the above rotation formula, it is not very difficult to implement it.

First, calculate the coordinates of the four corners after rotation (with the center of rotation as the coordinate origin)

Const double cosangle = cos (angle); const double sinangle = sin (angle); // The coordinates of the four corners of the original image are changed to point2d lefttop (-center. x, center. y); // (0, 0) point2d righttop (SRC. cols-center. x, center. y); // (width, 0) point2d leftbottom (-center. x,-Src. rows + center. y); // (0, height) point2d rightbottom (SRC. cols-center. x,-Src. rows + center. y); // (width, height) // coordinates of the four corners after center rotation point2d translefttop, transrighttop, transleftbottom, transrightbottom; translefttop = coordinates (lefttop, angle ); transrighttop = coordinates (righttop, angle); transleftbottom = coordinates (leftbottom, angle); transrightbottom = coordinates (rightbottom, angle );

Note that the coordinates of the four corners of the original image must be changed to the coordinates of the coordinate system with the center of rotation as the coordinate origin. Then, by rotating the transformation formula

Obtain the coordinates of the four angles after rotation.

The positions of the four angles after different rotation angles are different from those of the original image. That is to say, the upper left corner of the original image is not necessarily the upper left corner of the rotated image, it may be in the lower right corner. Therefore, after calculating the image width after rotation, you cannot use the horizontal coordinates after the rotation in the upper right corner of the source image minus the horizontal coordinates after the rotation in the lower left corner of the original image. The same is true for the height. (When searching for data, we found that most of them calculate the width and height of the image using this method ).

// Calculate the width, height double left = min ({translefttop. x, transrighttop. x, transleftbottom. x, transrightbottom. x}); double right = max ({translefttop. x, transrighttop. x, transleftbottom. x, transrightbottom. x}); Double top = max ({translefttop. y, transrighttop. y, transleftbottom. y, transrightbottom. y}); double down = min ({translefttop. y, transrighttop. y, transleftbottom. y, transrightbottom. y}); int width = static_cast <int> (ABS (left-right) + 0.5); int Height = static_cast <int> (ABS (top-down) + 0.5 );

To calculate the width of the rotated image, you can use the horizontal coordinates of the rightmost vertex after the four angles are rotated minus the horizontal coordinates of the leftmost vertex. in height, the vertical coordinates of the top vertex minus the vertical coordinates of the bottom vertex.

Then, you can use the final rotation formula to process each pixel of the image.

Const double num1 =-ABS (left) * cosangle-ABS (top) * sinangle + center. x; const double num2 = ABS (left) * sinangle-ABS (top) * cosangle + center. y; vec3b * P; For (INT I = 0; I 
The interpolation method used here is the nearest interpolation. The Implementation Method of bilinear interpolation is similar to that of image scaling.
 
Using the above algorithm for image rotation, we will find that the final result is the same regardless of the position in the image as the center of rotation. This is because the size of the image after rotation is the same for different positions as the center of rotation. The difference is only the position. In the last Transformation (three coordinate transformations are used for image rotation), the coordinates are uniformly moved to the origin.Rotated image. This is equivalent to performing a translation on the image, moving its position together, and finally rotating the image.
If, after the rotation, the coordinate origin is not moved to the upper left corner of the image after the rotation, but the upper left corner of the original image, what would happen?

Just like, some areas of the image will be truncated. Of course, if the rotation center is different, the final image will be different, and the truncated part will be different.
3. Combined Transformation
Combination transformation is to put multiple types of ry together. The conversion formula of rotation is deduced above, so the combination transformation is not very difficult. It is nothing more than multiplication of several matrices. A common combination transformation: scaling, rotation, and translation. The following example describes the formula for the combination transformation.

 

 
  
Zoom
Set (x0, y0) to the scaled coordinate, (x, y) to the scaled coordinate, and SX and Sy to the scaled factor.

 
  
Translation
(X0, y0) is the coordinate after translation, (x, y) is the coordinate before translation, dx, Dy is the offset

 
  
Rotate
(X0, y0) is the coordinate after rotation, (x, y) is the coordinate before rotation, (m, n) is the center of rotation, a is the angle of rotation, (left, top) is the coordinate of the upper left corner of the rotated image


 
Three transformation matrices are obtained respectively, which are combined according to the order of scaling, translation, and rotation.

Attention should be paid to the sequence during combination transformation. After all, the left multiplication and right multiplication of the matrix are different.
 
4. Last
The longest article ever written. It's almost a week since I put it together with the previous article. In addition to a few setbacks in the rotation of the algorithm, several other transformations have been smooth. Time-consuming mainly refers to drawing and mathematical formulas. Drawing has never found a proper tool, but it is gone. mathematical formulas have spent a day learning latex, some tools have been found to convert Tex to HTML, but the effects are not very good, or paste the image. Without graphs, it is hard to say clearly that some things rely only on words. It is time to learn MATLAB plotting.
 
 

 
 
 
 

 
Opencv2: geometric transformation of images, translation, mirroring, scaling, and rotation (2)