Initial knowledge of AdaBoost algorithm

Source: Internet
Author: User

1. The algorithm idea is simple: AdaBoost is an iterative algorithm, the core idea is to train different classifiers for the same training set, that is, weak classifiers, and then set up these weak classifiers to construct a stronger final classifier. (Three Stooges, the top of Zhuge Liang)

Its adaptation is that the sample of the previous basic classifier is strengthened, and the weighted sample is again used to train the next basic classifier. At the same time, a new weak classifier is added to each round until a predetermined small enough error rate is reached or a predetermined maximum number of iterations is reached.

2. Specifically, the entire AdaBoost iterative algorithm is 3 steps:

    1. Initializes the weight distribution of the training data. If there are N samples, each training sample is given the same weight at the very beginning: 1/n.
    2. Training weak classifiers. In the training process, if a sample point has been accurately classified, then in the construction of the next training set, its weight is reduced, conversely, if a sample point is not accurately classified, then its weight is improved. Then, the weight-updated sample set is used to train the next classifier, and the entire training process goes on so iteratively.
    3. The weak classifiers of each training are combined into strong classifiers. After the training process of each weak classifier is finished, the weight of the weak classifier with small classification error rate is enlarged, which plays a larger role in the final classification function, while the weight of the weak classifier with large classification error rate is reduced, which plays a smaller role in the final classification function. In other words, a weak classifier with a low error rate occupies a larger weight in the final classifier, otherwise it is smaller.

3. We can summarize some of the scenarios that are actually available for the adaboost algorithm:

1) Application scenarios for two classification or multi-classification

2) Baseline for classification tasks

No brain, simple, no overfitting, no need to tune the classifier

3) for Feature selection (feature selection)

4) The boosting framework is used to modify the Badcase

Only need to add new classifier, do not need to change the original classifier

Because the AdaBoost algorithm is a simple implementation, the application is also very simple algorithm. The AdaBoost algorithm obtains the strong classifier by combining the weak classifier, and the upper bound of the classification error rate decreases steadily with the increase of training, and does not cross the nature of the fitting and so on, it should be said to be an algorithm which is suitable for the application in various classification scenarios.

Initial knowledge of AdaBoost algorithm

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.