Boosting and Bagging

First, describe bootstraps: it can be considered as a sampling method with replacement.

Bagging: boostraps aggregating (Summary)

Boosting: Specifies the method for prompting the consumer T (Adaptive boosting ).

Boosting: In classification, you can change the weight of the training sample, learn multiple classifiers, and linearly combine these classifiers to improve the classification performance.

Thought: The tip method is to start from the weak learning algorithm, learn repeatedly, get a series of weak classifiers (base classifiers), and then combine these weak classifiers to form a strong classifier

1) Change the probability distribution of training data 2) change the distribution of data weights.

AdaBoost: 1) Increase the sample weights that are not correctly classified and lower those that are correctly classified. In this way, the samples with incorrect classification will be more concerned by the classifier later.

2) weighted majority voting is adopted to increase the classifier weights with high accuracy.

Algorithm steps:

(1) initialize the weight

(2) The basic classifier GM (X) is obtained by learning the training data with the weight distribution DM ).

(3) Calculate the classification error rate of GM (X) on the Training dataset.

(4) Calculate the coefficient of GM (X)

(5) Update the weight distribution. (If the weight is increased due to a classification error, the weight must be reduced if the classification is correct (I will not enter the formula ))

Instead of changing the training data, the distribution of weights of the data is constantly changing, so that the training data plays a different role in the learning of the basic classifier.

(6) loop until multiple classifiers are trained.

(7) construct a basic Classifier

 

Later, I learned Python and implemented it myself. This is serial and related to the learning and training results of the previous classifier ., Sampling Based on Error Rate.

 

Bagging:

By using Bootstrap to randomly select a training dataset, the training modelers of the classifier are independent of each other and can adopt parallel methods.

The basis of bagging is repeated sampling. By generating repeated Bootstrap instances of samples as the training set, each time the data of the same size as the sample is randomly selected from the total sample (not necessarily the same size ), yes, so some sample data may be repeated. However, this is actually used. In this way, a differentiated classifier is generated by increasing the differences in training data to improve the generalization ability of integration.

I don't want to talk about this theoretical analysis. I don't understand it ..