Computer vision and Pattern recognition (2)--a4 paper rectification

The last time I wrote the edge extraction of A4 paper, I found that there are still a lot of problems in my code, such as the criticism of the static threshold value, as well as the unstructured programming style. So I re-organized, the A4 paper Edge extraction code into a class. But that damn threshold, I haven't found a perfect way to make it work for all the images _ (: З"∠) _.

The method of optimization is one thing, that is to reduce the standard, merit . That is, the threshold is set very low, but the results of peak extraction only take the best 4. Of course, this method occasionally takes on strange edges, and because of the decrease of the threshold, the computational amount also increases exponentially, especially the Hough transform. But in a comprehensive sense, robustness has increased considerably.

In addition, you can search for " edge extraction dynamic threshold acquisition " of the paper. A simple method of gradient threshold dynamic is to take the average of all pixel gradient, as to how the effect, waiting for everyone to try Oh ~

So what else can we do on the basis of the previous section?

Now we just know the edges and corners of the A4 paper, and each piece of A4 paper is at different angles and positions, even with different shapes. These "deformed" A4 papers are not conducive to further image processing, so they need to be corrected into a uniform rectangle.

input Image:

Plain A4 printing paper may have handwritten notes or printed content, but may not be angled when taking pictures.

Output Image:

The standard plain A4 paper (210:297), which has been corrected, is cut out of other useless content and retains only the full A4 paper.

In the experiment, I used two methods, one is projective Transform(affine transformation), and the other is morphing(deformation).

Both methods have advantages and disadvantages, in which the affine transformation processing speed is fast, the correction picture is accurate, but involves the matrix calculus. Morphing image accuracy is not high affine transformation, but the principle of easy to understand, do not need calculus.

method One: projective Transform

Main reference this blog: http://blog.csdn.net/xiaowei_cqu/article/details/26471527

But there are several errors that need to be pointed out, not 0 o'clock, which should be obtained:

A11= x1-x0 +A13* X1 A12= y1-y0 +A13* Y1 A13=

A21 = x3-x0 + A23 * x3 A22= y3-y0 +A23* y3 A23=

A31 = x0 a32 = y0 A33 = 1

Key code:

matrix3x3 a4shapecorrect::squaretoquadrilateral (double x0, double y0, double x1, double y1, double x2,double y2, double x3 , double y3) {Double dx3 = x0-x1 + x2-x3;double dy3 = y0-y1 + y2-y3;if (dx3 = = 0.0f && dy3 = 0.0f) {Matri x3x3 result (x1-x0, y1-y0, 0,  x2-x1, y2-y1, 0,  x0, y0, 1); return result;  else {Double dx1 = x1-x2;double DX2 = x3-x2;double dy1 = y1-y2;double Dy2 = y3-y2;double denominator = dx1 * Dy2- DX2 * dy1;double A13 = (DX3 * DY2-DX2 * dy3)/denominator;double A23 = (DX1 * dy3-dx3 * dy1)/denominator;  matrix3x3 result (x1-x0 + A13 * x1, y1-y0 + a13*y1, A13, x3-x0 + a23*x3, Y3-y0 + a23*y3, A23,  x0, y0, 1); return Result;}}

After calculating the coefficients of the transformation matrices, it is only necessary to apply them to each pixel coordinate.Note the direction: map from the target pixel to the original pixel, and calculate the interpolation.

matrix3x3 H = squaretoquadrilateral (dots[0]->x, Dots[0]->y, Dots[1]->x, Dots[1]->y,    dots[2]->x,  Dots[2]->y, Dots[3]->x, dots[3]->y);/* Method 1:projective transforming */cimg_forxy (*target, x, y) {double _x = (double) x/_width;double _y = (double) y/_height;double denominator = h.a13 * _x + h.a23 * _y + h.a33;double tx = (h.a1 1 * _x + h.a21 * _y + h.a31)/denominator;double ty = (h.a12 * _x + h.a22 * _y + h.a32)/Denominator;cimg_forc (*target, c) (*target) (x, y, 0, c) = Bilinear (img, TX, ty, 0, c);}

method Two: Morphing

The Morph transformation for triangles is similar to bilinear interpolation:

How to calculate the coordinates of P ':: da/ba = D ' A '/b ' a ', PC/DC = P ' C '/D ' C ', where ABC, A ' B ' C ' and P-coordinates are known.

With the method of calculating the position of the transform triangle, we can cut the original A4 paper diagonal into two triangles, and then cut the target graph into two triangles, and then make the morph transformation of the pixel coordinates in the two triangles respectively.

One of the problems to be solved is how to judge a dot within the triangle, reference blog: http://www.cnblogs.com/graphics/archive/2010/08/05/1793393.html

I used the 3rd method.

Key code:

dot* A4shapecorrect::morph (Triangle source, Triangle target, dot p) {dot a = Source.a, B = source.b, c = source.c;dot at =  TARGET.A, BT = target.b, CT = target.c;if (p = = a) return new dot (At.x, At.y), if (p = = b) return new dot (bt.x, bt.y); if (p = = c) return new Dot (Ct.x, CT.Y); Line CP (c, p); Line AB (A, b);D ot* d = cp.intersect (AB);//Da/badouble Abrate = (b.x-a.x! = 0)? (d->x-a.x)/(b.x-a.x): (D->Y-A.Y)/(B.Y-A.Y);//Pc/dcdouble Cdrate = (d->x-c.x! = 0)? (p.x-c.x)/(d->x-c.x): (P.Y-C.Y)/(D->Y-C.Y);d ouble DTX = abrate* (bt.x-at.x) + at.x;double dty = abrate * (BT.Y-AT.Y) + at.y;double ptx = cdrate* (dtx-ct.x) + ct.x;double pty = cdrate* (DTY-CT.Y) + Ct.y;return new Dot (PTX, pty);}

Also note the direction: mapping from the target pixel to the original pixel, and calculating the interpolation.

/* Method 2:morphing */dot at (0, 0);D ot Bt (Target->width ()-1, 0);D ot Ct (Target->width ()-1, Target->height () -1);D ot Dt (0, Target->height ()-1);D ot A (dots[0]->x, dots[0]->y);D ot B (dots[1]->x, dots[1]->y);D ot C ( Dots[2]->x, dots[2]->y);D ot D (dots[3]->x, dots[3]->y); Cimg_forxy (*target, x, y) {Dot P (x, y), *p = Null;if (PO Intintriangle (Triangle (AT, BT, CT), p)) p = Morph (Triangle (at, BT, CT), Triangle (A, B, C), p); else if (Pointintriangle (Tria Ngle (AT, CT, dt), p)) P = morph (for Triangle (AT, CT, dt), Triangle (A, C, D), p), if (P! = NULL) {Cimg_forc (*target, C) (*target) (x , y, 0, c) = Bilinear (img, p->x, p->y, 0, C);}}

For both of these methods, there is a premise: we find the original A4 paper corner points must be in order ABCD arrangement, or at least we know the angle of the corresponding A4 paper orientation.

My method is 4 points first ordered by the y-axis, y is not the smallest point A is the B point, and then find the nearest point, and found that the two points are A and B (in fact, not necessarily, such as flat diamond). Then determine whether the rectangle is horizontal or vertical, and then accurately determine the position of AB, and finally accurately determine the CD location. Can only say so far so good!

void A4shapecorrect::reorderdots (std::vector<dot*>& dots) {Std::sort (Dots.begin (), Dots.end (), [] (const dot* A, const dot* b) {return a->y < b->y;});  Double min = dbl_max;int temp;for (int i = 1; i < dots.size (); ++i) {Double dis = dots[0]->distance (*dots[i]); if (min > Dis) {min = Dis;temp = i;}} Std::swap (Dots[1], dots[temp]); if (Dots[1]->y > Dots[2]->y | | dots[1]->y > Dots[3]->y) {std::swap ( Dots[0], dots[1]); if (Dots[2]->y > Dots[3]->y) {std::swap (dots[2], dots[3]);}} Else{if (Dots[0]->x > Dots[1]->x) {std::swap (dots[0], dots[1]);} if (Dots[2]->x < dots[3]->x) {Std::swap (dots[2], dots[3]);}}}

Computer vision and Pattern recognition (2)--a4 paper rectification