Happened to come across a blog XGBoost vs LightGBM: How Are They Different. Let's investigate a bit wider and deeper into the following 4 machine learning open source packages.
- XGBoost: XGBoost Doc, XGBoost Source Code
- LightGBM: LightGBM Doc, LightGBM Source Code
- CatBoost: CatBoost Doc, CatBoost Source Code
- H2O: H2O Doc, H2O Source Code
- MLflow: MLflow Doc, MLflow Source Code
1. Some Readings
- XGBoost vs LightGBM: How Are They Different
- Gradient Boosted Decision Trees [Guide]: a Conceptual Explanation
- AdaBoost and Gradient Boost – Comparitive Study Between 2 Popular Ensemble Model Techniques
- AdaBoost Vs Gradient Boosting: A Comparison Of Leading Boosting Algorithms
- A Primer to Ensemble Learning – Bagging and Boosting
- Ensemble methods: bagging, boosting and stacking
- CatBoost Classifier in Python
- Tutorial: CatBoost Overview
Cited from XGBoost vs LightGBM: How Are They Different
1 | Two of the most popular algorithms that are based on Gradient Boosted Machines are XGBoost and LightGBM. |
2. Concepts
- Bootstrap Aggregating: also named Bagging. A typical Bagging method is Random Forest.
- Boosting: A typical Boosting method is AdaBoost, short for Adaptive Boosting.
- Ensemble: As summarized, Bagging, Boosting, and Stacking ALL belong to Ensemble.
- Cascading: citing:
Unlike voting or stacking ensembles, which are multiexpert systems, cascading is a multistage one. - Gradient Boosting: citing:
originated in the observation by Leo Breiman, an American academician, also proposed Random Forest.
3. Amazon.com - Employee Access Challenge
In this blog, I'm going to use a very old Kaggle dataset Amazon.com - Employee Access Challenge.
4. XGBoost
Please make sure: - either rebuild your own XGBoost - or download my built wheel from https://longervision.us/
1 | ➜ ~ pip show xgboost |
5. LightGBM
Please make sure: - either rebuild your own LightGBM - or download my built wheel from https://longervision.us/
1 | ➜ ~ pip show lightgbm |
Pay Attention: In order to have XGBoost or LightGBM successfully built, you may have to: - either rebuilt your own CuPy - or download my built wheel from https://longervision.us/
6. CatBoost
1 | ➜ ~ pip show catboost |
7. H2O
1 | ➜ ~ pip show h2o |
8. MLflow
1 | ➜ pip show mlflow |
Please make sure: - either rebuild your own MLflow - or download my built wheel from https://longervision.us/