Automatic license plate recognition system based on ' matching ' technology

With the rise of intelligent parking system, license plate recognition technology is really a fire. I'm here, too. Interest, a general view of the current situation, found that the current popular license plate recognition technology is based on neural networks (see Mastering OpenCV with practical computer Vision projects book, Issued by the mechanical industry press). Artificial neural network So advanced things, I do not understand, but as the image matching technology background, I think this small case needs so complex to complete it? and training Artificial Neural network is also a troublesome thing ... In short, under a variety of unconvinced, the author based on the image matching technology to write a license plate automatic recognition algorithm, self-test dozens of images, I feel the effect is good, so take out show off, I hope you criticize.

First explain the development environment, I based on OpenCV2.4.10, with VS2010 development, with the use of WINGSL math library, these things on the Internet, I do not say much. Next.

I designed the license plate recognition algorithm can be broadly divided into 7 large steps, of which 3 of the main steps are used in the image matching technology, the following steps:

1, color-based license plate segmentation to get two value of the image, this step is mainly based on the majority of the license plate is a blue background of the characteristics (non-standard non-blue license plate recognition, need to be relatively complex, this article is not discussed).

Step1:segment Blue background/////// void SegmentBackground2 (mat& img, mat& Binary) {binary.create (2, Img.size, CV_8UC1); unsigned char *psrcdata, * pbindata;int width = img.cols;int height= img.rows ; unsigned char* data = Binary.ptr (); memset (data, 0xFF, binary.step * height); int i, j;double total = 0.0f;int counter = 0; unsigned char blue, green, red;for (i = 0; i < height; + + i) {psrcdata = Img.ptr (i);p bindata = Binary.ptr (i); int k = 0;f or (j = 0; J < width; + + j) {blue = psrcdata[k];green= psrcdata[k + 1];red  = psrcdata[k + 2];d ouble Ave = (blue + gr Een + red)/3.0f;if (blue * 0.75 > Green && green * 0.80 > Red && Ave >) {pbindata[j] = 0;++ C Ounter;total + = blue;} K + = 3;}}}

       2. The results of the segmentation are connected to the region and contour tracking, the minimum bounding box is calculated according to the contour for each labeled area, and the independent regions are sorted according to the area of connected area. Then from large to small, according to certain criteria to start merging these connected areas, and finally determine the license plate area.

int labelbackgroundimg (mat& binary, mat& labelimg, std::vector<int>& area, Std::vector<ccontour >& excontours) {iplimage iplbinary = binary;labelimg.create (2, Binary.size, cv_32s); int* Plabel = (int*) ( LABELIMG.DATA); int num = Labelimagewithexcontours (&iplbinary, Plabel, excontours); Area.assign (num, 0); int w = Binary.cols;int h = binary.rows;for (int i = 0; i < h; + + i) {int* Labeldat = (int*) (Labelimg.ptr (i)); for (int j = 0; j < W; + + j) {if (Labeldat[j] > 0) {int n = labeldat[j]-1;++ area[n];}}} return num;} void Unitecomponets (std::vector<int>& area, std::vector<ccontour>& excontours, RotatedRect&  box, std::vector<int>& labels) {int n2 = excontours.size (); int nn = Area.size (); int* ids = new int[nn];int* Area_ = new Int[nn];for (int i = 0; i < nn; + + i) {ids[i] = i;area_[i] = Area[i];} for (int i = 0; I < nn-1, + + i) {for (int j = i + 1; j < nn; + + j) {if (Area_[i] < area_[j]) {int t = Area_[i];are A_[i] = AREA_[J];AREA_[J] = T;t = Ids[i];ids[i] = ids[j];ids[j] = t;}}} Delete[] Area_;int id = ids[0]; Cpointset ptset;ptset.resize (Excontours[id]. Getlenth ()); Excontours[id]. Exportpointset (ptset); int orgptnum = Ptset.getpntscount (); cvpoint* orgpts = new Cvpoint[orgptnum];p tset.exportdata (orgpts); Mat Orgptmat (1, Orgptnum, CV_32SC2, orgpts); Rotatedrect Orgbox = Minarearect (Orgptmat); Labels.clear (); Labels.push_back (ID); for (int i = 1; i < nn; + + i) {int id = i Ds[i];p tset.resize (Excontours[id]. Getlenth ()); Excontours[id]. Exportpointset (ptset); int ptnum = Ptset.getpntscount (); cvpoint* pts = new Cvpoint[orgptnum + ptnum];p tset.exportdata (pts + orgptnum); Mat Ptmat (1, Ptnum, CV_32SC2, pts + orgptnum); Rotatedrect box = Minarearect (Ptmat); memcpy (pts, orgpts, sizeof (cvpoint) * orgptnum); Mat ptmattotal (1, orgptnum+ ptnum, CV_32SC2, pts); Rotatedrect boxtotal = Minarearect (ptmattotal), if (Boxtotal.boundingrect (). Area () < Orgbox.boundingrect (). Area () + Box.boundingrect (). Area () * 1.5) {labels.push_back (id); oRgbox = Boxtotal;orgptnum + = Ptnum; cvpoint* temp = orgpts;orgpts = Pts;pts = temp;} Delete[] pts; Delete[] orgpts;delete[] Ids;box = Orgbox;}

3. Re-get the two-value image of the license plate area.

void Setbinaryimage (mat& binary, mat& lable, int labelstotalnum, std::vector<int> platelabels) {unsigned  char* labelmap = new unsigned char[labelstotalnum + 1];memset (labelmap, 0xFF, Labelstotalnum + 1); for (unsigned int i = 0; I < platelabels.size (); + + i) {int id = platelabels[i] + 1;//Because the token is starting from 1, and the array subscript starts with 0 labelmap[id] = 0x00;}  unsigned char* bDat = Binary.data;memset (BDat, 0xFF, Binary.step * binary.rows); int id;for (int i = 0; i < binary.rows;  + + i) {int* Ldat = (int*) (Lable.ptr (i)), BDat = Binary.ptr (i), for (int j = 0; J < Binary.cols; + + j) {id = ldat[j];if (id > 0) bdat[j] = Labelmap[id];}} Delete[] Labelmap;}

4. The image is normalized to solve the problem of different image size. Get an image of the same size plate area.

void Normalizeimage (rect& Rect, rotatedrect& box, mat& binaryimg, mat& plateimg, rotatedrect& NewBox , mat& newbinaryimg, mat& newplateimg) {Rect arerect = Box.boundingrect ();d ouble ratio = 300.0/arerect.width;if ( 75.0/arerect.height > ratio) ratio = 75.0/arerect.height;int imgsize[2];imgsize[0] = Int (rect.height* ratio + 100.5f ); imgsize[1] = Int (rect.width * ratio + 100.5f), newbox.center.x = float (imgsize[1]/2.0), Newbox.center.y = float (imgsize[ 0]/2.0); newbox.angle = Box.angle;newbox.size.width = Float (box.size.width * ratio); newbox.size.height= Float (box.size . Height * ratio); point2f srcpnts[3];srcpnts[0].x = float (rect.x), srcpnts[0].y = float (RECT.Y); srcpnts[1] = srcpnts[0];srcpnts[1].x + = RECT.WIDTH;SRCPNTS[2] = srcpnts[0];srcpnts[2].y + = Rect.height; POINT2F dstpnts[3];d stpnts[0].x = 50;DSTPNTS[0].Y = 50;dstpnts[1] = dstpnts[0];d stpnts[1].x + = float (rect.width * ratio); DSTPNTS[2] = dstpnts[0];d stpnts[2].y + = float (rect.height* ratio); Mat TRANSMat = Getaffinetransform (srcpnts, dstpnts);//Is there an easy way to find this transformation? Size dstsize (imgsize[1], imgsize[0]); Newbinaryimg.create (2, Imgsize, Binaryimg.type ()); newbinaryimg = Scalar (255, 0, 0 ); Warpaffine (binaryimg, newbinaryimg, Transmat, Dstsize, Inter_nearest, border_transparent);// Whether the resize function can be substituted for newplateimg.create (2, Imgsize, Plateimg.type ()), newplateimg = Scalar (255, 255, 255); Warpaffine ( Plateimg, Newplateimg, Transmat, Dstsize, Inter_linear, border_transparent);//Whether resize functions can be used instead of}//step2.2:normalize Binary image blocks//////////////////////////////////////////////////////////////////////////void Dilate_ Erodebinaryimg (mat& binaryimg) {int height= binaryimg.rows;int width = binaryimg.cols; Mat dist;distancetransform (binaryimg, Dist, CV_DIST_L2, cv_dist_mask_precise); int i, j;for (i = 0; i < height; + + i) {FL oat* Distdat = (float*) (Dist.ptr (i)), unsigned char* Bindat = (unsigned char*) (Binaryimg.ptr (i)); for (j = 0; j < width; + + j) {if (Distdat[j] < 7.5) Bindat[j] = 0;}} Dilate (binaryimg, BinaRyimg, Mat (), point (-1,-1), 3);} 

5. Through the image matching technology, obtain the correction image under the perspective projection of the license plate area.

BOOL Rectifyplateimage (mat& Img, mat& binary, rotatedrect& box, mat& rectifiedimg) {Mat edge = Binary.clon E (); Canny (binary, Edge, 50, 120, 5);//The Mat Iedge;edge.convertto (Iedge, 1, -1.0, 255) can be implemented in a simple way,//for debugimwrite (_t ("f:\\ My VC10 application \\plate num rec\\test\\test_contour_img.bmp "), Iedge); Mat dist;distancetransform (Iedge, Dist, CV_DIST_L1, cv_dist_mask_precise); for (int i = 0; i < dist.rows; + + i) {float* D at = dist.ptr<float> (i), for (int j = 0; J < Dist.cols; + + j) Dat[j] = sqrt (dat[j]);} int w = img.cols;int h = img.rows; POINT2F verts[4];box.points (Verts); POINT2F Normalverts[4]; Getmatchvertexs (Verts, Normalverts); normalverts[0].x-= 5; NORMALVERTS[0].Y-= 5;normalverts[1].x + = 5; NORMALVERTS[1].Y-= 5;normalverts[2].x + = 5; Normalverts[2].y + = 5;normalverts[3].x-= 5; Normalverts[3].y + = 5; Fitrectangle (Dist, normalverts); POINT2F orgverts[4];float dd = 3;orgverts[0].x = 32-DD;ORGVERTS[0].Y = 32-dd;orgverts[1].x =288 + dd;orgVerts[1].y = 3 2-dd;orgverts[2].x =288 + dd;orgverts[2].y = + + dd;orgverts[3].x = 32-DD;ORGVERTS[3].Y = + + dd; Mat Transmat = Getperspectivetransform (Normalverts, orgverts); Size SIZEDST (+/-); Warpperspective (IMG, rectifiedimg, Transmat, SIZEDST); return true;} void Getmatchvertexs (point2f* verts, point2f* normalverts) {int indexs[4];d ouble MAXMV = verts[0].x * verts[0].y;double mi  NMV = verts[0].x * verts[0].y;double MAXDV = verts[0].x/verts[0].y;double MINDV = Verts[0].x/verts[0].y;memset (Indexs,  0x00, sizeof (int) * 4); for (int i = 1; i < 4; + + i) {double tempm = verts[i].x * verts[i].y;double tempd = verts[i].x/ Verts[i].y;if (tempm > MAXMV) {maxmv = tempm;indexs[2] = i;} else if (TEMPM < MINMV) {MINMV = tempm;indexs[0] = i;} if (Tempd > Maxdv) {maxdv = tempd;indexs[1] = i;} else if (TEMPD < MINDV) {MINDV = tempd;indexs[3] = i;}} for (int i = 0; i < 4; + + i) normalverts[i] = Verts[indexs[i]];}

6. Through image matching technology, further segmentation of the image region of each character,

int getbluespacepoints (mat& templateimg, point2d* templatepnts) {int h = templateimg.rows;int W = templateImg.cols; int cc = 0;FOR (int i = 0; i < h; + + i) {unsigned char* Pdat = templateimg.ptr (i); for (int j = 0; J < W; + + j) {if (PD AT[J] = = 0) {templatepnts[cc].x = J;templatepnts[cc].y = i;++ cc;}}} return cc;}  BOOL Bluespacematch (mat& image, point2d* templatepnts, int ncount, double Val, point2d Center, double& Scaleratio, double& centershiftx, double& centershifty) {double a_b[3];d ouble deltax[3];d ouble al[3];d ouble MM[9]; point2d* ppntset = new point2d[ncount];memcpy (Ppntset, templatepnts, sizeof (POINT2D) * ncount);d ouble* PL = new Double[nCo UNT];d ouble* pA = new Double[ncount * 3];for (int j = 0; J < ncount; + + j) {ppntset[j].x-= center.x;ppntset[j].y-= CEN TER.Y;} Double va0, VA1, Va2, VA3, va4;double sx, sy;double perror;perror = 0.0f;a_b[0] = scaleratio;a_b[1] = centershiftx;a_b[2] = centershifty;for (int i = 0; i <; + + i) {for (int j = 0; J < NcounT + + j) {SX = a_b[0] * ppntset[j].x + a_b[1];sy = a_b[0] * ppntset[j].y + a_b[2];va0 = getdatavalue (image, SX, sy); va1 = Ge Tdatavalue (image, SX + 0.5, sy), va2 = Getdatavalue (image, sx-0.5, sy), va3 = Getdatavalue (image, SX, SY + 0.5); va4 = getd Atavalue (image, SX, sy-0.5);d ouble dx = va1-va2;double DY = va3-va4;pa[j] = DX * ppntset[j].x + DY * PPNTSET[J].Y;PA [j + 1 * ncount] = dx;pa[j + 2 * ncount] = dy;//Apl[j] = val-va0;//L = g-g (')}double error = 0.0;for (int j = 0; J &L T ncount; + + j) Error + = pl[j] * PL[J];IF (i = = 0) perror = error;if (Perror < error) {for (int i = 0; i < 3; + + i) {Deltax[i] *= 0 .5;a_b[i]-= Deltax[i];} int j;for (j = 0; J < 3; + + j) {if (Fabs (Deltax[j]) > 0.0000001) break;} if (j = = 3) break;continue;} perror = Error;int noff_m = 0;for (int j = 0; J < 3; + + j) {for (int JJ = 0; JJ < 3; + + JJ) {mm[noff_m + JJ] = 0.0;for (i NT KK = 0; KK < ncount; + + KK) Mm[noff_m + JJ] + = pa[j * ncount + KK] * PA[JJ * ncount + KK]; }//M = A * A (T) noff_m + = 3;} GslexInversematrix (MM, 3);/m = m (-) int j;for (j = 0; J < 3; + + j) {Al[j] = 0.0;//AL = A * lfor (int kk = 0; KK < ncount ; + + KK) Al[j] + + pa[j * ncount + KK] * PL[KK];} for (j = 0; J < 3; + + j) {Deltax[j] = 0.0;for (int jj = 0; JJ < 3; + JJ) Deltax[j] + mm[j * 3 + JJ] * AL[JJ];} DeltaX = (A * A (t)) (-) * A (t) * lfor (j = 0; J < 3; J + +) {if (Fabs (Deltax[j]) > 0.0000001) break;} if (j = = 3) break;for (j = 0; J < 3; j + +) A_b[j] + = deltax[j];//A = a + delta/b = B + delta}delete[] pl;delete[] pa;scal Eratio = A_B[0];CENTERSHIFTX = A_b[1];centershifty = a_b[2];d elete [] Ppntset;return true;} int Getcharblockrect (double& scaleratio, double& centershiftx, double& centershifty, Rect* rects) {Size Templateimgsize;templateimgsize.width = 256;templateimgsize.height= 64;//point2d center;center.x = Templateimgsize.width/2;center.y = TEMPLATEIMGSIZE.HEIGHT/2; Rect Templateblockrect[7];int Xx[7] = {1, 7,, Templateblockrect[i, 155, 225};for (int i = 0; i <; + + i) .x = xx[i];templateblockrect[i].y = 4;templateblockrect[i].width = 29;templateblockrect[i].height= 57;} for (int i = 0; i < 7; + + i) {templateblockrect[i].x-= Int (center.x + 0.5f); templateblockrect[i].y-= Int (center.y + 0.5f); rects[i].x = Int (templateblockrect[i].x * scaleratio + CENTERSHIFTX + 0.5f); rects[i].y = Int (Templateblockrect[i] . Y * scaleratio + centershifty + 0.5f), rects[i].width = Int (templateblockrect[i].width * scaleratio + 0.5f); Rects[i].heigh t= Int (templateblockrect[i].height* scaleratio + 0.5f);} return 7;} int extractcharblockimg (mat& plateimg, rect* rects, mat* blockimgs) {Size dstsize;dstsize.width = 30;dstSize.height= 60;for (int i = 0; i < 7; + + i) {int size[2];size[0] = dstsize.width;size[1] = dstsize.height;blockimgs[i].create (2, Siz E, Plateimg.type ()); Mat temp (plateimg, rects[i]); Resize (temp, blockimgs[i], dstsize);} return 7;}

7. Identify individual characters through image matching techniques.

int Recognizeword (Inputarray blockimg); int Recognizechar (Inputarray blockimg); int Recognizenumchar (InputArray BLOCKIMG); int recognizeblockimg (mat& blockimg, int i) {if (I < 0 | | i > 6) return-1; Mat Grayimg;cvtcolor (blockimg, grayimg, Cv_rgb2gray); Mat Grayimginv;grayimg.convertto (GRAYIMGINV,-1,-1.0, 255); Mat Binaryimg;adaptivethreshold (GRAYIMGINV, binaryimg, 255, Adaptive_thresh_gaussian_c, THRESH_BINARY, 51, 9); Mat Binaryimgsmooth; Gaussianblur (binaryimg, Binaryimgsmooth, Size (3, 3), 1.25f, 1.25f); Imwrite (_t ("f:\\ my VC10 application \\plate num rec\\test\\ Test_block_binary_char.bmp "), binaryimgsmooth), int nres = -1;if (i = = 0) nres = Recognizeword (Binaryimgsmooth); else if (i = = 1) nres = Recognizechar (Binaryimgsmooth); Else nres = Recognizenumchar (Binaryimgsmooth); return nres;} void Matchtemplatesword (Inputarray templatsimg_, Inputarray charimg_, std::vector<double>& results); void Matchtemplateschar (Inputarray templatsimg_, Inputarray charimg_, std::vector<double>& resuLTS); void Matchtemplatesnumchar (Inputarray templatsimg_, Inputarray numimg_, std::vector<double>& results) ; int Recognizeword (Inputarray blockimg) {Mat tempimg = Imread (_t (". \\template\\word_grayScale_AutoSize_Smooth.bmp "), Cv_load_image_grayscale);std::vector<double> results; Matchtemplatesword (tempimg, blockimg, results);d ouble minval = results[0];unsigned int nl = 0;for (unsigned int i = 1; I & Lt Results.size (); + + i) {if (Results[i] < minval) {minval = RESULTS[I];NL = i;}} return NL;} int Recognizechar (Inputarray blockimg) {Mat tempimg = Imread (_t ("... \\template\\char_grayScale_AutoSize_Smooth.bmp "), Cv_load_image_grayscale);std::vector<double> results; Matchtemplateschar (tempimg, blockimg, results);d ouble minval = results[0];unsigned int nl = 0;for (unsigned int i = 1; I & Lt Results.size (); + + i) {if (Results[i] < minval) {minval = RESULTS[I];NL = i;}} return NL;} int Recognizenumchar (Inputarray blockimg) {Mat tempimg = Imread (_t ("... \\template\\num_char_grayScale_AUtosize_smooth.bmp "), Cv_load_image_grayscale);std::vector<double> results; Matchtemplatesnumchar (tempimg, blockimg, results);d ouble minval = results[0];unsigned int nl = 0;for (unsigned int i = 1; I < results.size (); + + i) {if (Results[i] < minval) {minval = RESULTS[I];NL = i;}} return NL;}

Code is more, some of the code does not affect the main line is not posted out. If everyone is interested in discussing can private messages I ha. Qiu Qiu 31667460

Automatic license plate recognition system based on ' matching ' technology