Solving the Problem of Overfitting

The Problem of Overfitting : article

Linear Regression/Logistic Regression:

弱弱	剛好	吃太補
underfit	just right	overfit
High bias	just right	high variance

Overfitting: If we have too many features, the learned hypothesis may fit the training ser very well, but fail to generalize to new examples.

Addressing overfitting:

1. Reduce number of features
   • Manually select which features to keep.
   • Model selection algorithm.
2. Regularization
   • Keep all the features, but reduce magnitude/ value of parameters θj.
   • Works well when we have a lot of features, each of which contributes a bit to predicting y.