The general idea of license plate recognition and Verification code recognition

Http://www.pin5i.com/showtopic-22246.html

Describe the ideas and algorithms.

The full text is divided into two parts, the first part of the recognition of the license plate recognition and common method of identification, the second part of the similar QQ verification code, Gmail verification code of this kind of abnormal authentication code recognition methods and ideas.

First, the license plate/Verification Code identification common method

The common methods for identification of license plate and verification code are:

(1) Grayscale and two value of the image

(2) de-noising, then cut into a single character

(3) Extracting characteristics of each character, generating a feature vector or feature matrix

(4) Classification and learning. The feature vectors or feature matrices are compared with the sample library, and the similar samples are selected, and the values of such samples are used as the output results.



The following code describes the above process. Because the update SVN Server, I used to BDB stored code access, so some of the code is reflector back to compile, I hope forgive.

(1) Grayscale and two value of the image

The purpose of this is to turn each pixel of the picture into 0 or 255 so that it can be calculated. Part of the noise can also be removed.

Image grayscale and two value of the premise is BMP picture, if not, you need to convert to BMP picture first.

Speaking in code, my image grayscale code (algorithm is searched on the Internet):

1. 1 protected static color Gray (color c)
2. 2 {
3. 3 int rgb = Convert.ToInt32 ((double) (((0.3 * C.R) + (0.59 * C.G)) + (0.11 * c.b));
4. 4 return Color.FromArgb (RGB, RGB, RGB);
5. 5}
6. 6

Copy Code

By grayscale the image, each pixel becomes a grayscale value of 0-255.

The gray value of two is then converted to 0 or 255. The general approach is to set a range, for example, [A, b], the gray level between [A, a] is all changed to 255, the other becomes 0. I am using an adaptive binary algorithm that is popular on the internet.

2. 1 public static void Binarizate (Bitmap map)
4. 2 {
6. 3 int TV = Computethresholdvalue (map);
8. 4 int x = map. Width;
10. 5 int y = map. Height;
12. 6 for (int i = 0; i < x; i++)
14. 7 {
16. 8 for (int j = 0; J < y; J + +)
18. 9 {
20. if (map. GetPixel (i, J). R >= TV)
22. 11 {
24. Map. SetPixel (i, J, Color.FromArgb (0xFF, 0xFF, 0xff));
26. 13}
28. + Else
30. 15 {
32. Map. SetPixel (i, J, Color.FromArgb (0, 0, 0));
34. 17}
36. 18}
38. 19}
40. 20}
42. 21st
44. private static int Computethresholdvalue (Bitmap img)
46. 23 {
48. int i;
50. int k;
52. Csum Double;
54. thresholdvalue int = 1;
56. int[] ihist = new int[0x100];
58. (i = 0; i < 0x100; i++)
60. 30 {
62. ihist = 0;
64. 32}
66. gmin int = 0xFF;
68. Gmax int = 0;
70. for (i = 1; i < (IMG). WIDTH-1); i++)
72. 36 {
74. The PNS for (int j = 1; J < (IMG). HEIGHT-1); J + +)
76. 38 {
78. The int cn = img. GetPixel (i, J). R
80. ihist[cn]++;
82. if (CN > Gmax)
84. 42 {
86. Gmax = CN;
88. 44}
90. if (CN < gmin)
92. 46 {
94. Gmin = CN;
96. 48}
98. 49}
100. 50}
102. Wuyi double sum = csum = 0.0;
104. int n = 0;
106. (k = 0; k <= 0xFF; k++)
108. 54 {
110. Sum + = k * Ihist[k];
112. + + + + ihist[k];
114. 57}
116. if (n = = 0)
118. 59 {
120. return 60;
122. 61}
124. Double Fmax =-1.0;
126. n1 int = 0;
128. (k = 0; k < 0xFF; k++)
130. 65 {
132. N1 + = Ihist[k];
134. if (n1! = 0)
136. 68 {
138. N2 int = n-n1;
140. if (N2 = = 0)
142. 71 {
144. Thresholdvalue return;
146. 73}
148. Csum + = k * Ihist[k];
150. M1 double = Csum/(double) n1);
152. Double m2 = (sum-csum)/((double) n2);
154. A double SB = ((n1 * n2) * (M1-M2)) * (M1-M2);
156. if (SB > Fmax)
158. 79 {
160. Fmax = SB;
162. Bayi Thresholdvalue = k;
164. 82}
166. 83}
168. 84}
170. Thresholdvalue return;
172. 86}
174. 87
176. 88

Copy Code

Grayscale and two before the value of the picture:



Grayscale and two after the value of the picture:





Note: for license plate recognition, this algorithm is good. For Authenticode identification, you may need to design a special two-valued algorithm for specific Web sites to filter the noise.

(2) de-noising, then cut into a single character

Above this license plate cutting is relatively simple, from left to right scan, encounter empty big, click a knife, solved. But there are some license plates, like this one:





A simple scan will not work. Therefore, a more general denoising and cutting algorithm is needed. Here I am using a relatively simple method:

Consider the picture above as a plane. The picture is projected horizontally, so that the projection value of the place with the word is high, and the value of the projection from the place where there is no word is low. This will get a curve, like one hill after another. Here is my hand drawing:




Then, a scan line (s in) is scanned from the bottom up. This scan line will have an intersection with the curve in the graph, which divides the hill into one area after another. The license plate image is generally 7 characters, so when the scan line divides the hill into seven areas, it stops. Then the seven characters in the picture can be divided according to the coordinates of the projection of the seven regions to the horizontal line.

But the reality is complicated. For example, the "Chuan" word, its horizontal projection is three hills. Press the scan method above to cut it. Therefore, for the above cut, the constraints need to be added: Each hill has a central line, the distance between the hill and the center of the hill must be above a certain value, otherwise, you need to merge the two hills. With this constraint, it is possible to cut effectively.

The above is a horizontal projection. Then we need to do vertical projection and cutting. Here the vertical projection with cutting on a hill, so good to handle some.

The results of the cut are as follows:



Horizontal projection and Cutting code:

2. 1 public static ilist<bitmap> Split (Bitmap map, int count)
4. 2 {
6. 3 if (count <= 0)
8. 4 {
10. 5 throw new ArgumentOutOfRangeException ("Count must be greater than 0.");
12. 6}
14. 7 ilist<bitmap> resultlist = new list<bitmap> ();
16. 8 int x = map. Width;
18. 9 int y = map. Height;
20. Ten int splitbitmapminwidth = 4;
22. int[] Xnormal = new Int[x];
24. (int i = 0; i < x; i++)
26. 13 {
28. + for (int j = 0; J < y; J + +)
30. 15 {
32. if (map. GetPixel (i, J). R = = Chargrayvalue)
34. 17 {
36. xnormal++;
38. 19}
40. 20}
42. 21}
44. Pair pair = new pair ();
46. (int i = 0; i < y; i++)
48. 24 {
50. ilist<pair> pairlist = new List<pair> (count + 1);
52. (int j = 0; J < x; j + +)
54. 27 {
56. if (Xnormal[j] >= i)
58. 29 {
60. if ((j = = (x-1)) && (pair. Status = = Pairstatus.start))
62. 31 {
64. The pair. End = j;
66. The pair. Status = Pairstatus.end;
68. if (pair. End-pair. Start) >= splitbitmapminwidth)
70. 35 {
72. Pairlist.add (pair);
74. 37}
76. The pair = new pair ();
78. 39}
80. The-else if (pair. Status = = pairstatus.justcreated)
82. 41 {
84. A pair. Start = j;
86. The pair. Status = Pairstatus.start;
88. 44}
90. 45}
92. The + else if (pair. Status = = Pairstatus.start)
94. 47 {
96. The pair. End = j;
98. The pair. Status = Pairstatus.end;
100. if (pair. End-pair. Start) >= splitbitmapminwidth)
102. 51 {
104. Pairlist.add (pair);
106. 53}
108. The pair = new pair ();
110. 55}
112. if (Pairlist.count > Count)
114. 57 {
116. break;
118. 59}
120. 60}
122. if (Pairlist.count = = Count)
124. 62 {
126. Pairlist foreach (Pair p in)
128. 64 {
130. P.width < (map. WIDTH/10))
132. 66 {
134. int width = (map. WIDTH/10)-p.width;
136. P.start = Math.max (0, P.start-(WIDTH/2));
138. P.end = math.min ((int) (P.end + (WIDTH/2)), (int) (map). WIDTH-1));
140. 70}
142. 71}
144. Pairlist foreach (Pair p in)
146. 73 {
148. newmapwidth int = (P.end-p.start) + 1;
150. Bitmap Newmap = new Bitmap (newmapwidth, y);
152. * for (int ni = P.start; ni <= p.end; ni++)
154. 77 {
156. (int nj = 0; nj < y; nj++)
158. 79 {
160. Newmap.setpixel (Ni-p.start, NJ, map. GetPixel (NI, NJ));
162. 81}
164. 82}
166. Resultlist.add (NEWMAP);
168. 84}
170. Resultlist return;
172. 86}
174. 87}
176. Resultlist return;
178. 89}
180. 90

Copy Code

The pair in the code represents a pair of intersections of the scan line with the curve:

1. 1 Private class Pair
2. 2 {
3. 3 public Pair ();
4. 4 public int Charpixelcount {get; set;}
5. 5 public int charpixelxdensity {get;}
6. 6 public int End {get; Set }
7. 7 public int Start {get; Set }
8. 8 public bitmapconverter.pairstatus Status {get; Set }
9. 9 public int Width {get;}
10. 10}
11. 11

Copy Code

Pairstatus represents the state of the pair. The specific state of what the meaning, I have forgotten.

1. 1 Private enum Pairstatus
2. 2 {
3. 3 justcreated,
4. 4 Start,
5. 5 End
6. 6}
7. 7

Copy Code

The above code is hard to write because there are a lot of special cases to deal with. The pairstatus was also introduced for the handling of special cases.

The code for vertical projection and cutting is simple, not affixed, see the attached DLL's Bitmapconverter.trimheight method. The

above uses a simple method of denoising and cutting. Some images, especially CAPTCHA images, require special noise removal. How to do this is to open cximage (http://www.codeproject.com/KB/graphics/cximage.aspx), or Paint.NET, with the above image processing methods, to see whether effective denoising. Remember your own steps, and then turn over their source code to extract the algorithm. What more refinement ah, filtering ah, these processing can improve the quality of the picture. Refer to the ITK code or image processing book for details.



(3) extracts the characteristics of each character, generates a feature vector or feature matrix

splits the cut character into a small block, such as 3x3,5x5, or 3x5, or 10x8, and then counts the number of pixels per small block with a value of 255 , so that a matrix M is obtained, or the matrix is simplified to vector v.

with the above 3 steps, the characters in a license plate can be numerically converted into vectors. The

(1)-(3)-Step-specific code flow is as follows:

1. 1
2. 2 bitmapconverter.tograybmp (bitmap); Grayscale of pictures
3. 3 bitmapconverter.binarizate (bitmap); Picture two value of
4. 4 ilist<bitmap> maplist = Bitmapconverter.split (Bitmap, Defaultcharscount); Horizontal projection and cutting
5. 5 Bitmap map0 = Bitmapconverter.trimheight (maplist[0], defaultheighttrimthresholdvalue); Vertical projection and cutting
6. 6 Imagespliter spliter = new Imagespliter (MAP0);
7. 7 Spliter. Widthsplitcount = Defaultwidthsplitcount;
8. 8 Spliter. Heightsplitcount = Defaultheightsplitcount;
9. 9 Spliter. Init ();
10. 10

Copy Code

Then, through Spliter. ValueList can obtain a vector representation of the Bitmap map0.

(4) Classification

The principle of classification is very simple. Use (VIJ,CI) to represent a sample. Among them, Vij is the vector of the sample image after its numerical process. CI is a human eye to recognize this picture, given the results. Vij shows that there are multiple samples, their numerical vectors are different, but their results are CI. Assuming that the image to be identified is vectorized, the resulting vector is V '.

Intuitively, we will have the idea that this image is to be identified, most like a picture in a sample library, then we will treat it as that image, identifying it as the pre-specified character of the image in the sample library.

In our eyes, it's easy to tell if a picture is similar to another picture, but it's hard for a computer to judge. We have already converted the image into a vector of the same dimension, how does a computer determine that a vector is similar to another vector?

Here you need to calculate the distance between a vector and another vector. The shorter the distance, the more similar the two vectors are considered.

I use samplevector<t> to represent vectors:

1. 1 public class Samplevector<t>
2. 2 {
3. 3 protected t[] Vector {get; set;}
4. 4 public Int32 Dimension {get {return vector.length;}}
5. 5 ...
6. 6}
7. 7

Copy Code

T represents a data type, can be Int32, or it can be a more precise type such as double.

The public interface for measuring distances is: IMetric

1. 1 public Interface Imetric<telement,treturn>
2. 2 {
3. 3 Treturn Compute (samplevector<telement> v1, samplevector<telement> v2);
4. 4}
5. 5

Copy Code

Commonly used is the minkowskimetric.

2. 1//<summary>
4. 2//Minkowski measure.
6. 3//</summary>
8. 4 public class Minkowskimetric<telement>: Imetric<telement, double>
10. 5 {
12. 6 Public Int32 Scale {get; private set;}
14. 7 Public Minkowskimetric (Int32 scale)
16. 8 {scale = scale;}
18. 9
20. Ten public Double Compute (samplevector<telement> v1, samplevector<telement> v2)
22. 11 {
24. if (V1 = = NULL | | v2 = = NULL) throw new ArgumentNullException ();
26. if (v1. Dimension! = v2. Dimension) throw new ArgumentException ("V1 and v2 dimensions vary.");
28. Double result = 0;
30. for (int i = 0; i < v1. Dimension; i++)
32. 16 {
34. Result + = Math.pow (Math.Abs (convert.todouble (v1)-convert.todouble (v2)), scale);
36. 18}
38. return Math.pow (result, 1.0/scale);
40. 20}
42. 21}
44. 22
46. Metricfactory is responsible for producing various dimensions of the minkowskimetric:
48. 24
50. public class Metricfactory
52. 26 {
54. public static imetric<telement, double> createminkowskimetric<telement> (Int32 scale)
56. 28 {
58. Return to new minkowskimetric<telement> (scale);
60. 30}
62. 31
64. public static imetric<telement, double> createeuclideanmetric<telement> ()
66. 33 {
68. Return createminkowskimetric<telement> (2);
70. 35}
72. 36}
74. 37

Copy Code

Minkowskimetric is a universally used measure. But not necessarily the most effective amount. Because it treats every point in vector v equally. In image recognition, the importance of each point is not the same, for example, the recognition of Q and O, features in the lower half, the lower part of the weight should be greater than the upper half. For these easily confusing characters, special measurement methods need to be designed. In license plate recognition, others are easily confused by D and 0,0 and O,i and 1. Minkowski metric recognize these characters, and the effect is poor. Therefore, special handling is required when these characters are encountered. As time was tight, I only used Minkowski Metric.

In my code, only the most recently implemented, whichever is selected. A better solution is to use K nearest neighbor classifier or neural network classifier. K Nearest neighbor principle is to find and to identify the picture (vector) distance from the nearest K samples, and then let the K sample using a certain rule calculation (vote), which category (C) of the new picture, the neural network will be measured process and voting decision process parameterization, so that it can be changed with the increase of the sample, Is such a kind of learning machine. Interested to see the "pattern Classification" of the Book of the third chapter and the fourth chapter.

The general idea of license plate recognition and Verification code recognition