[Pattern recognition] boosting

Boosting

In classification, multiple weak classifiers are usually combined into a strong classifier for classification, collectively referred to as the integrated classification method (ensemble method ). This method is simpler than Boosting. For example, if you use the bagging method before boosting, You can first sample samples from the population sample set to train a weak Classifier in different training sets, and then use multiple weak classifiers for voting, the final result is the superior result of classifier voting. Such a simple voting policy is usually difficult to achieve very good results. It was not until the boosting method came out that the power of weak classifier combinations was realized. Boosting means to strengthen and improve, that is, to promote a weak classifier to a strong classifier. We often hear that AdaBoost is the most representative of boosting in the future. The so-called Adaboost, that is, adaptive boosting, refers to the adjustment of the error rate of the weak classifier based on the learning result feedback adaptively. Therefore, no prior knowledge is required for autonomous training. Breiman praised AdaBoost as the best off-the-shelf method in his paper.

Two types of discrete adaboos algorithm flows

The merge sting methods include discrete Adaboost, real Adaboost, logitboost, and gentle AdaBoost. The framework for training all methods is similar. Take discrete AdaBoost as an example. The training process is as follows:

First, initialize the same weight for each sample (step 2). Then, use the weighted sample to train each weak classifier (Step 3.1 ); the weighted training error rate and proportional factor (Step 3.2) are obtained after classification. The weights of the samples with incorrect classification are increased and the modified weights are normalized again (Step 3.3 ); in the cycle training process, a weak classifier is combined with a proportional factor to form a final strong classifier.
The combination process and results of multiple weak classifiers are roughly as follows:

It is an effective method to accelerate training by increasing the weight of the samples that are incorrectly classified during the training cycle. Because the weak classifier with a high accuracy rate in training has a large weight, more and more samples are being correctly classified in the new round of training. Training Samples with a low weight play a small role in the new round of training, that is, each round of new training focuses on training samples with misclassification.

In actual training, the weak classifier is the same, but the actual training data used by the weak classifier is different. Generally, each dimension of the feature vector is used to form a weak classifier. Later, the well-known Haar + AdaBoost face detection method uses each Haar feature to form a weak classifier. Block-based Haar features have a lot of other information than simple pixel-based features, generally, we can get better checking results, and the integral graph integral method also has a great advantage in computing speed. If you are interested, you can refer to Face Recognition Based on AdaBoost and Haar-like features.

Real AdaBoost and gentle merge stdiscrete AdaBoost are the simplest two types of boosting classification results, and the real Adaboost (also called Adaboost. mh) can be seen as the generalization form of discrete AdaBoost. The weak classifier can output multiple classification results and output the possibility of these classification results. It can be seen that every weak classifier is not "arbitrary ". Gentle AdaBoost is a method that modifies the wrong sample weight adjustment during iterative training, with less emphasis on difficult samples to be classified, in this way, the efficiency of the classifier is reduced due to the high adjustment of positive sample weights of "atypical" by AdaBoost ., The generated variant algorithm. AdaBoost's MATLAB e toolkit gml_javasst_matlab _toolbox implements real Adaboost, gentle Adaboost, and modest Adaboost, and provides an overview (the Toolkit uses internal hands-on failover, you can also take the exam to the next article "cart and GML AdaBoost MATLAB toolbox"):

As for logitadaboost, I actually don't know much about it. For details, I can refer to "opencv's notes on AdaBoost".

(Reprinted please indicate the author and Source: http://blog.csdn.net/xiaowei_cqu do not use for commercial purposes without consent)

[Pattern recognition] boosting