Comparison of LIGHTGBM and Xgboost

At the beginning of 2017, Microsoft Open source New machine learning Framework LIGHTGBM, based on GBDT, it is said that the Higgs dataset LIGHTGBM nearly 10 times times faster than Xgboost, the memory occupancy rate is about 1/6 of Xgboost
English Document: http://lightgbm.apachecn.org/cn/latest/index.html

Xgboost is the best boosting model of the past, since rumors of LIGHTGBM performance is higher, then what is the difference between them.

First, the motives proposed by LIGHTGBM:
Commonly used machine learning algorithms, such as neural network algorithms, can be trained in mini-batch way, the size of training data will not be limited by memory.

In each iteration, GBDT needs to traverse the entire training data multiple times. If the entire training data is loaded into memory, it will limit the size of the training data, and if you do not load the memory, read and write the training data repeatedly will consume a very large amount of time. Especially in the face of industrial-grade massive data, the ordinary GBDT algorithm can not meet their needs.

LIGHTGBM The main reason is to solve the GBDT in the massive data encountered problems, so that GBDT can be better and faster for industrial practice.

Ii. the difference between LIGHTGBM and xgboost:
1. Xgboost uses a decision tree algorithm based on the pre-ordering method (pre-sorted), and LIGHTGBM uses a decision tree algorithm based on the histogram (histogram). The pre-sorting algorithm needs to calculate the gain of the split once every time a eigenvalue is traversed, while the histogram algorithm only has to calculate k times (k can be considered constant), and time complexity is optimized from O (data*feature) to O (k*features)
Note:
2, xgboost using the growth strategy of the decision tree by layer Growth (level-wise), LIGHTGBM using the depth limit of the leaf growth (leaf-wise) algorithm, in the same number of split times, leaf-wise can reduce more errors, To get better accuracy. The disadvantage of leaf-wise is that it is possible to grow a more deep decision tree and produce overfitting.
3, support the category characteristics, do not need to single-hot code processing
4. Optimization of feature parallelism and data parallel algorithm

In summary: LIGHTGBM accuracy and xgboost similar, even slightly above xgboost, and training speed is much higher than xgboost