Data mining algorithm Learning (5) C4.5

C4.5 classification decision tree algorithm, whose core algorithm is ID3. It is currently used in clinical decision-making, manufacturing, document analysis, bioinformatics, spatial data modeling, and other fields. Algorithm input is data with a class object, and output is a decision-making rule in the tree.

C4.5 is better than ID3:

1) The information gain rate is used to select attributes, which overcomes the shortcomings of attributes with many options;
2) pruning during tree construction;
3) discretization of continuous attributes;
4) ability to process incomplete data.

Advantage of C4.5 algorithm: The generated classification rules are easy to understand and have high accuracy.

Disadvantages of The C4.5 algorithm: During tree construction, datasets need to be scanned and sorted multiple times, which leads to inefficient algorithms.

Algorithm process:

? C4.5 (dataset, featurelist):-create the root node R-if the data in the current dataset belongs to the same class, mark the r class as this class-if the current featurelist set is empty, the class marked with r is the class with the largest number of samples in the current dataset-recursion :? Select the attribute F from featurelist (select the maximum attribute of gainratio (dataset, f). For continuous attributes, see the discretization process above )? Divide dataset into different subsets of DS based on each value V of F. For each DS:-create node C-If DS is empty, node C is the class with the largest number of samples in dataset-If DS is not empty, node c = C4.5 (DS, featurelist-f)-add node C as a subnode of R

WEKA is used to construct a decision tree using C4.5 classification for weather data, for example:

Use the SQL algorithm to implement the core code of The C4.5 algorithm:

drop procedure if exists buildtree;DELIMITER |create procedure buildtree()begindeclare le int default 1;declare letemp int default 1;declare current_num int default 1;declare current_class varchar(20);declare current_gain double;declare current_table varchar(20) default 'weather';update infoset set state=1,statetemp=1;rr:while (1=1) doset @weather = (select play from weather where class is null limit 0,1);set @feature =(select x from infoset where statetemp=1 limit 0,1);if (@weather is null ) thenleave rr;else if(@feature is null) thenupdate infoset set statetemp = state; end if;end if;if (@weather is not null) thenb:beginset current_gain = (select max(info_Gain) from infoset where statetemp=1);set current_class = (select x from infoset where info_Gain = current_gain);drop table if exists aa;set @a=concat('create temporary table aa select distinct ',current_class,' as namee from weather where class is null');prepare stmt1 from @a;execute stmt1;tt:while (1=1) doset @x = (select namee from aa limit 0,1);if (@x is not null) thena0:begindrop table if exists bb;set @b=concat('create temporary table bb select * from ', current_table,' where ',current_class,' = ? and class is null');prepare stmt2 from @b;execute stmt2 using @x;set @count = (select count(distinct play) from bb);if (@count =1) thena1:beginupdate weather set class = current_num,levelnum = letemp where id in (select id from bb); set current_num = current_num+1;if (current_table ='cc') thendelete from cc where id in (select id from bb);end if;set @f=(select play from cc limit 0,1);if (@f is null) thenset current_table='weather';update infoset set statetemp=state; set letemp =le;end if;delete from aa where namee = @x;end a1;end if;if (@count>1) thendrop table if exists cc;create temporary table cc select * from bb;set current_table = 'cc';set letemp = letemp+1;leave tt;end if;if(@count=0) thendelete from aa where namee = @x;set le = le+1;end if;end a0;else update infoset set state=0 where x=current_class;leave tt;end if;end while;update infoset set statetemp=0 where x=current_class;  end b;end if;end while;end |delimiter ;

Shows the classification result after running:

 

The class attribute records the classification result, and the levelnum attribute records the layers of each data in the decision tree.

Description of tables in the program:

? Table 1 weather adds the levelnum attribute column to store the layers of each record in the generated category tree .? Table 2 c45_temp is used to store intermediate results. The attribute columns logv and parent are used to store the intermediate calculation result and category of each attribute component.
? Table 3 infoset stores the information gain rates of each attribute. The attribute columns state and statetemp are used to mark during calculation.

Data mining algorithm Learning (5) C4.5