aiacademy: 機器學習 Ensemble Methods

Ensemble

用系統化的方式，將好幾個 supervised learning 的model 結合再一起，試圖將結果截長補短，變成一個更強的model !!

• Ensemble methods

  • Bagging: resample training data

    • Random forest

  • Boosting: reweight training data

    • AdaBoost

    • Gradient Boosting

  • Stacking: blendding weak learners

Bagging

• Bootstrap
  • random sampling with replacement

AdaBoost

• Boosting

  

Gradient boosting

• 老師推薦好書:
  • The Elements of Statistical Learning

• Gradien boosting vs random forest

  • Random forset generates many trees; these trees are independent to each other

  • Gradient boosting many trees one by one, the new trees try to correct to predictions of previous trees

Stacking

Summary

• Ensemble to imrove the base learners

• Bagging : resample training data

  • Random forest: 每一棵樹只能看到某些features，看到些features？也是random決定的

• Boosting: iteratively create new models to compensate the old models

  • AdaBoost, gradient boosting

• Stacking: blending weark learners