less than 1 minute read

Tags: , ,

Ensemble

用系統化的方式,將好幾個 supervised learning 的model 結合再一起,試圖將結果截長補短,變成一個更強的model !!

  • Ensemble methods

    • Bagging: resample training data

      • Random forest
    • Boosting: reweight training data

      • AdaBoost

      • Gradient Boosting

    • Stacking: blendding weak learners

Bagging

  • Bootstrap
    • random sampling with replacement

AdaBoost

  • Boosting

    Imgur

Gradient boosting

  • Gradien boosting vs random forest

    • Random forset generates many trees; these trees are independent to each other

    • Gradient boosting many trees one by one, the new trees try to correct to predictions of previous trees

Stacking

Imgur

Summary

  • Ensemble to imrove the base learners

  • Bagging : resample training data

    • Random forest: 每一棵樹只能看到某些features,看到些features?也是random決定的
  • Boosting: iteratively create new models to compensate the old models

    • AdaBoost, gradient boosting
  • Stacking: blending weark learners