Opencv2.4.9 Source Analysis--cascade Classification (III)

The first two articles respectively introduced the Cascade classifier principle and source code analysis, below we give a concrete application example.

Below we take the license plate recognition as an example, specifically explains the use of OPENCV cascade classifier. Here we only on the blue bottom white on the ordinary license plate identification, for the other license plate is not considered in the range. and the license plate is positive, slightly inclined, tilt is too large is not in the recognition range.

We collected a total of 1545 photos of the required license plate images through different channels (unfortunately, I can only get so many license plate photos, if more can be better!) ), the license plate image is cut from the photo by hand by ACDSee software and saved in JPG format. To facilitate subsequent processing, we name the filenames in numerical order, as shown in 8. Then we save the license plate images in the POS folder.

Fig. 8 The image of the Blue bottom white license plate

It is important to note that here we do not need to scale the license plate image to a uniform size (that is, the size of the positive sample image), it is not necessary to convert them to grayscale images, these work can be completed by the system. We just need to tell the system license plate image file, the location of the license plate, and the size of the license plate.

In order to do this efficiently, we have written the following code:

#include "opencv2/core/core.hpp" #include "opencv2/highgui/highgui.hpp" #include "opencv2/imgproc/imgproc.hpp" # Include <iostream> #include <fstream> #include <string>using namespace cv;using namespace Std;int main (int argc, char** argv)    {ofstream postxt ("Pos.txt", ios::out); Create the Pox.txt file if (!postxt.is_open ()) {cout<< "Can not creat POS txt file!";    return false;} n represents the total number of license plate images, C indicates the number of license plate sample images that can eventually be used int N = 1545, c = 0; int width, height, i; String filename;    Mat posimage;for (i=0;i<n;i++)//traverse all license plate images {filename = to_string (i) + ". jpg";    Get the current license plate image filename Posimage = imread ("pos\\" + filename); Open the current license plate image if (Posimage.empty ()) {cout<< "Can not open" + filename + "file!" <<endl;continue;}    width = posimage.size (). width;    The width height of the current license plate image = Posimage.size (). Height; The current license plate image high//if the current license plate image width is less than 60, or higher than 20, then reject the license plate image if (Width < | | Height <) {cout<<filename + "too small!"        <<endl;continue;} Writes information about the current license plate image to the Pos.txt file postxt<< "Pos/"+ filename +" 1 0 0 "+ to_string (width) +" "+ to_string (height) <<endl;c++;    Cumulative}cout<<c;    Terminal output C value postxt.close (); Close Pos.txt file return 0;}

After executing the program, the output from the terminal C value is 1390, which indicates that 155 (1545-1390) license plate images are removed due to size too small. In addition, in the current directory we also got the Pos.txt file, which is exactly what the system needs, it's file content 9 is shown.

Figure 9 Pos.txt File

In the Pos.txt file, each row represents an image file. We take the first example, it represents a 0.jpg file in the Pos folder, the following "1" means that the file has only one sample image (that is, the license plate), and then the "0 0" in the back of the sample image to indicate the upper-left coordinate, because we have cut the image, each JPG file is a complete license plate, So the three variables of all rows are "1 0 0". The last "450 140" indicates the width and height of the 0.jpg file.

We collected 10589 images without a license plate, no watermark, no logo, no date photos. These photos are converted uniformly to JPG format, and are also named in the order of numbers, as shown in 10. Then we put these photos in the Neg folder.

Figure 10 Photo with no license plate image

The size of these photos is not required, as long as the size of the image is larger than the normal sample, because the system is to cut these photos, so as to obtain a positive sample image size of the same negative sample image, so a picture can be a number of negative sample images. These photos are as diverse as possible, and the content of each photo is as rich as possible, but the most important thing is that it cannot contain license plate information.

We also need to provide the system with a text file that holds these photo information. In the same way, we have written a simple program to do the job:

#include "opencv2/core/core.hpp" #include "opencv2/highgui/highgui.hpp" #include "opencv2/imgproc/imgproc.hpp" # Include <iostream> #include <fstream> #include <string>using namespace cv;using namespace Std;int main (int argc, char** argv) {ofstream negtxt ("Neg.txt", ios::out);    Create Neg.txt file if (!negtxt.is_open ()) {cout<< "Can not creat neg txt file!"; return false;}    n represents the total number of photos, and C indicates the number of resulting photos int N = 10589, c=0;int i; String filename; Mat posimage;for (i=0;i<n;i++) {filename = to_string (i) + ". jpg";    Photo filename Posimage = imread ("neg\\" + filename);    Open the current photo if (Posimage.empty ()) {cout<< "Can not open" + filename + "file!" <<endl;continue;} negtxt<< "neg/" + filename<<endl;    Write photo file name C + + to neg.txt files;    Cumulative}cout<<c;    Terminal output C value negtxt.close ();    Close Neg.txt file return 0;}

After executing the program, a neg.txt file is obtained in the current directory, which is shown in file content 11.

Figure Neg.txt File

When the above content is ready, we can use the relevant program provided by OPENCV to get a cascade classifier that can identify the license plate.

First, create a new plate folder under D, and we'll save the Pos and neg with a large number of photo images previously mentioned, As well as Pos.txt and neg.txt, copy the text files to the plate folder and create a new data folder within the plate folder (required later). Because my computer is a 64-bit Win7 system, the compiler uses Microsoft Visual Studio 2012 and therefore needs to replicate from within the Opencv/build/x64/vc11/bin folder Opencv_ Createsamples.exe and Opencv_traincascade.exe These two files into the plate folder. The Opencv_createsamples.exe is used to create a positive sample VEC file required by the system, and Opencv_traincascade.exe is used to train the cascading classifier. All two files need to be run under the command line.

Opencv_createsamples.exe need more parameters, here we only have to use the parameters to explain:

-info: Used to represent a text file containing a license plate photo, i.e. Pos.txt

-BG: A text file that represents a photo without a license plate, i.e. Neg.txt

-vec: The positive sample VEC file name of the output, we name this file Pos.vec

-num: Number of license plate photo images, i.e. 1390

- w: Width of positive sample image (pixels)

- H: High (pixel) of the positive sample image

The latter two parameters need us to fill in according to the actual situation, because we only on the blue-white license plate identification, the actual size of this type of license plate is 440mmx140mm, we must maintain a positive sample image width and height is also this proportion, and the width and height can not be too large, not too small. In general, we choose:-W for 58,-h to 18.

Before we prepared the license plate photo, we did not scale the license plate to the size of 58x18, because the opencv_createsamples.exe will be uniformly scaled according to the two parameters of-W and-H, so there is no previous processing.

The final Opencv_createsamples.exe command is:

Opencv_createsamples.exe-info pos.txt-bg Neg.txt-vec pos.vec-num 1390-w 58-h 18

For the sake of convenience, we save this command to the Createsamples.bat batch file so that it can be executed as long as it executes. The execution results in 12, and the Pos.vec file is generated within the plate folder.

Figure Opencv_createsamples.exe Execution Results

The following is the implementation of Opencv_traincascade.exe to train the Cascade classifier, the command requires more parameters, but it is important, they have the following meanings:

-data: folder name, used to save the training generated by the various XML files, the folder must be created beforehand, otherwise the system will error, here, we define the folder named data, it has been created in the previous

-vec: A positive sample VEC file generated by the Opencv_createsamples.exe program, that is, Pos.vec

-BG: A text file that represents a photo without a license plate, i.e. Neg.txt

-numpos: Number of positive samples used to train each level classifier (i.e. strong classifier) of the Cascade classifier

-numneg: The number of negative samples used to train each level classifier (i.e. strong classifier) of the Cascade classifier

-numstages: Finally get the cascade classifier's progression, we set to 12

-precalcvalbufsize: The amount of memory space, in megabytes, used to store pre-computed eigenvalue values

-precalcidxbufsize: The amount of memory space used to store the pre-computed feature index in megabytes

-stagetype: The type of the strong classifier, which currently only implements the AdaBoost, so the unique value (the default) is boost

-featuretype: Feature type, HAAR (default), LBP or hog

- W: The width of the positive sample image must be consistent with the parameters of the Opencv_createsamples.exe command, i.e. 58

- H: The high of the positive sample image must be consistent with the parameters of the Opencv_createsamples.exe command, i.e. 18

-BT: Type of AdaBoost, dab,rab,lb or gab (default)

-minhitrate: Minimum recognition rate for each level classifier mentioned in the principle section

-maxfalsealarmrate: Maximum error rate per level classifier mentioned in the principle section

-weighttrimrate: Pruning for decision trees with a default value of 0.95

-maxdepth: The maximum depth of the decision tree, the default value is 1, that is, the decision tree is a two-tree (stump-shaped)

-maxweakcount: The maximum number of decision trees contained by the strong classifier, which is also related to the maximum error rate, we define this value to be 150

-mode: If the feature is Haar, this parameter determines which Haar feature to use (see Figure 1), BASIC (default), Core, or all

Here we will focus on the selection of several important parameters. Because my computer's memory is 16G, in order to maximize the use of this memory, we have-precalcvalbufsize and-precalcidxbufsize both parameters are defined as 5000, that is, 5G. The minimum recognition rate and the maximum error rate determine the length of the training time and the quality of the recognition, and we define the two values as 0.999 and 0.25, respectively. -numpos refers to the training of strong classifiers used in the number of positive samples, it is not the total number of positive samples, in principle, the larger the value, the better the quality of the classifier, but also to consider the recognition rate, if the recognition rate is not high, there will be some positive samples are identified as negative samples, so there must be some redundancy, Of course, the system also takes into account this point, that is, if the positive sample is exhausted, and has not reached the specified number of Numpos, then the system will adjust the value to the actual quantity (detailed information in the previous Source Analysis section). We set the value to 1300. The-numneg set to how big seems to be inconclusive, but by reading the original text of the Viola & Jones algorithm, they use 9,832 positive samples (4,916 face images, plus their vertical mirror image) and 10,000 negative samples, the number of positive and negative samples is close to 1:1, So we set Numneg to 1350.

The final Opencv_traincascade.exe command is:

Opencv_traincascade.exe-data Data-vec pos.vec-bg neg.txt-numpos 1300-numneg 1350-numstages 12-precalcval BufSize 5000-precalcidxbufsize 5000-w 58-h 18-maxweakcount 150-mode all-minhitrate 0.999-MAXFALSEALARMR ATE 0.25

In the same vein, we also save this command to the batch file Train.bat. It is also important to note that the case of the parameter must be partitioned, otherwise the system error.

Figure Opencv_traincascade.exe parameter information for output during execution

Figure Opencv_traincascade.exe Information of the 3rd-level strong classifier output during execution

When the command is executed, the terminal first outputs some parameter information, as shown in 13. Then there is the training information for each level of the strong classifier of the output cascade classifier, because we set the numstages to 12, so there are 12 strong classifiers: 0-stage to 11-stage. Figure 14 shows the information for the 3rd-level strong classifier. Here we analyze the meaning of the information:

===== TRAINING 3-stage =====

<begin

Indicates the beginning of the training of level 3rd strong classifiers.

POS count:consumed 1300:1302

When training this level of strong classifiers, 1300 positive sample images can be used, and the 1300 positive sample images are selected from 1302 positive sample images, i.e. two positive samples are not identified at this time. The previous 1300 is the number specified by the parameter Numpos in the Opencv_traincascade.exe command, and sometimes this value is less than Numpos, indicating that the Numpos setting is too large and the minimum recognition rate setting is smaller, resulting in an insufficient number of positive sample images. The following 1302 can be used to indicate the recognition rate of the current cascade classifier, that is, the recognition rate of the Cascade classifier composed of 0-stage, 1-stage, 2-stage. The recognition rate at this time is 99.846%, because 1300÷1302=0.99846.

NEG Count:acceptanceratio 1350:0.00620359

The ability to use 1350 negative sample images when training this high-level classifier is exactly the number specified by the parameter Numneg in the Opencv_traincascade.exe command, which may be less than Numneg, because the POS in the preceding message The value of count is not equal to Numpos, the size of the specific value of the source analysis. The following 0.00620359 represents the negative sample acceptance rate, which is the error rate of the Cascade classifier consisting of all strong classifiers (0-stage, 1-stage, 2-stage) before the current strong classifier, i.e., after the current cascade classifier is predicted, These 1350 images, which are predicted as positive samples and actually negative samples, are obtained from the number of negative sample images. The cascade classifier is characterized by the fact that the post-level strong classifier receives only those data that the pre-classifier considers to be positive samples, and predicts negative samples as positive samples, which will increase with the training progression, and the difficulty level is also increasing with opencv_ The maximum error rate set in the Traincascade.exe command is Maxfalsealarmrate, and the lower the error rate setting, the greater the degree of difficulty. Take this level as an example, these 1350 negative samples are selected from the shrank negative sample, the formula is: 1350÷0.00620359≈217615. In this case the last level of the strong classifier training, this value even up to 1 billion. So the time spent in the training process is mainly here. Before the line information is displayed, the terminal outputs the following information: NEG current SAMPLES:XXXX. XXXX represents the current time to get negative sample number, this value will gradually increase, when increased to 1350, it will normally display the above information. When the error rate of the cascaded classifier is lower than the error rate we set at this time (for example, now we have got 3 strong classifiers: 0-stage, 1-stage, 2-stage, we have to train the 4th strong classifier 3-stage, when this strong classifier is well trained, The maximum error rate that the Cascade classifier consisting of these 4 strong classifiers should satisfy is: 0.25x0.25x0.25x0.25=0.00390625), the system stops training because the current cascade classifier has already met the requirements and no further training is required.

Precalculation time:52.337

Represents the time taken to pre-compute the eigenvalues, i.e. we calculate a subset of the eigenvalues before the strong classifier is built, and the value is Opencv_ The arguments in the Traincascade.exe command are related to the precalcvalbufsize and precalcidxbufsize, which means that the more memory we have in advance for this purpose, the more values are saved, so the longer it takes to calculate the eigenvalues. The time to build a strong classifier is greatly shortened because the eigenvalues used are calculated before the strong classifier is built.

+------+-------------+-------------+

|   n|   HR | FA |

+------+-------------+-------------+

|       1|       1| 1|

+------+-------------+-------------+

|       2|       1| 1|

+------+-------------+-------------+

...... ......

+------+-------------+-------------+

| 10|0.999231|0.336296|

+------+-------------+-------------+

| 11|0.999231|0.228148|

+------+-------------+-------------+

n indicates the number of training of the weak classifier (i.e. decision tree) of the current strong classifier, HR indicates the recognition rate of the current strong classifier, and FA indicates the error rate of the current strong classifier. We start from the bottom line 2nd, at this time training to get 10 decision tree, the recognition rate is 99.9231%, the error rate is 33.6296%, the recognition rate satisfies the requirements, that is, the minimum recognition rate of 99.9%, but the error rate does not meet the requirements, that is, it is greater than the maximum error rate of 25%, so also need to continue training, When a decision tree is obtained (i.e. there are 11 decision trees at this time), both the recognition rate and the error rate meet the requirements (99.9231%>99.9%,22.8148%<25%).

End>

Indicates that the strong classifier at this stage has been obtained, because the recognition rate and error rate are satisfied, so the training of this level of strong classifier ends.

Training until now have taken 0 days 0 hours27 minutes 2 seconds.

Indicates that so far the training cascade classifier has been 27 minutes and 2 seconds.

Figure Opencv_traincascade.exe Command execution end

Figure 15 shows the entire cascade classifier after the completion of the training interface, you can see that training for more than 10 hours altogether. My computer's CPU is Intel Core i5-4690k. If we change the recognition rate and error rate to 0.9995 and 0.2 respectively, it will take more than a day, and if we adjust the progression to 13, it will take 6 days.

When the training is over, you get the Cascade.xml file in the Data folder, which is exactly the Cascade classifier data we need, and we can use it to identify the license plate.

The following program is a simple application:

#include "opencv2/core/core.hpp" #include "opencv2/highgui/highgui.hpp" #include "opencv2/imgproc/imgproc.hpp" # Include "opencv2/objdetect/objdetect.hpp" #include <iostream> #include <fstream> #include <string> Using namespace Cv;using namespace Std;int main (int argc, char** argv) {cascadeclassifier classifier ("Cascade.xml"); 
   
    //Instantiate cascading classifier mat img = imread ("car.jpg");    Read photos vector<rect> plates;    Represents the license plate area    //license plate recognition, the default identification of the minimum license plate is a positive sample of the area (here is 58x18), the largest area of the entire picture, that is, only to identify the area of 58x18 or more than the license plate Classifier.detectmultiscale (IMG, plates); for (int i = 0; i < plates.size (); i++)    //Draw the license plate area rectangle (IMG, plates[i], Scalar (255, 0, 255), 2); Imshow ("pl Ates ", IMG); waitkey (0); return 0;}
   

Figure 16 Recognition results

Figure 16 is the effect of the run. Because the license plate photo is not many, can not be a comprehensive measurement of the recognition effect, but never more experimental results, although the case of error detection, the detected license plate also has incomplete phenomenon, but basically can meet the requirements. I found that simply increasing the recognition rate or reducing the error rate and increasing the progression did not seem to improve the problem, and I think only increasing the number of positive samples is an effective way to improve the quality of recognition.

Here is the license plate recognition for the video file:

#include "opencv2/core/core.hpp" #include "opencv2/highgui/highgui.hpp" #include "opencv2/imgproc/imgproc.hpp" # Include "opencv2/objdetect/objdetect.hpp" #include <iostream> #include <fstream> #include <string>    Using namespace Cv;using namespace Std;int main (int argc, char** argv) {videocapture vedio ("Cars.avi"); Read Video if (!vedio.isopened ()) {cout<< "Video Open failed! "<<endl;return 1;}    Double rate= Vedio.get (cv_cap_prop_fps);    Get the frame frequency int delay= int (1000/rate); Define a delay time mat frame;    Size size = size (int (vedio.get (cv_cap_prop_frame_width)), int (Vedio.get (cv_cap_prop_frame_height)));    Size of the video image//define a write video file Videowriter writer ("Plates.avi", CV_FOURCC (' M ', ' J ', ' P ', ' G '), rate, size, true); if (!writer.isopened ()) {cout << "Initialize Videowriter failed! "<< Endl;return 1;}        Cascadeclassifier classifier ("Cascade.xml"); while (true) {if (!vedio.read (frame)) break;vector<rect> plates; License plate detection, here set the maximum size of the license plate is 190x60classifier.detectmultiscale (frame, plates, 1.1, 3, 0, SIze (), Size (p)); for (int i = 0; i < plates.size (); i++) Rectangle (frame, plates[i], Scalar (255, 0, 255), 2);//Plus text P Uttext (frame, "HTTP://BLOG.CSDN.NET/ZHAOCJ", point (50,60), Cv_font_hershey_complex,0.7,scalar (255,0,0), 2);    Writer.write (frame); Write Video if (Cv::waitkey (delay) >=0) break;} Vedio.release (); return 0;}

I uploaded the video results to the following URLs. The video is 3 minutes, can be seen in the license plate of the recognizable size range, can accurately identify the license plate, of course, also error detection and license plate recognition is not complete phenomenon:

Http://v.youku.com/v_show/id_XMjI4ODM3Mjk1Ng==.html

In addition, I also uploaded the Cascade.xml file to the following Web site, you can download the test:

http://download.csdn.net/detail/zhaocj/9737259

Opencv2.4.9 Source Analysis--cascade Classification (III)