1.id3 Selecting the properties for maximizing information gainC4.5 Selecting the properties for maximizing gain ratio
avoidance problem: ID3 preferences to divide data into many parts of the propertyResolution: Take into account the number of data sets after partitioning
| entropy |
(where rf-relative frequency)
|
information gain->id3
|
|
|
gain ratio->c4.5
|
|
when the data is divided into many parts, the proportion of each share becomes smaller, P (s,b) becomes larger, and the Gain ratio becomes smaller .
2.c4.5 The handling of missing valueA. When building the treeattribute Selectiondivided by attribute B, the data that is empty is marked as S0 and does not participate in the calculationwhen the empty value is the majority, G will be small, so that it is not biased to select more than the null value of the property
Only the information obtained from the non-empty part of the data is considered and multiplied by the weight |
|
|
|
Data partitioningProperty B is empty data, grouped into subclasses by scale of other classes Si class will get | s0|* each S0 data
B. The probability of each type of prediction of the leaf node is proportional to the prediction, for example (0.25,0.5,0.25) when it encounters a node and uses attribute B to divide it, and when the data y,b to be predicted is empty, the classification results of each subtree need to be traversed. and the number of data that is divided by each subtree is determined by how much weight
3. Pruning does not look clear ...
From for notes (Wiz)
The improvement of C4.5 more ID3
