Solving the Problem of Overfitting
The Problem of Overfitting : article
Linear Regression/Logistic Regression:
|High bias||just right||high variance|
Overfitting: If we have too many features, the learned hypothesis may fit the training ser very well, but fail to generalize to new examples.
- Reduce number of features
- Manually select which features to keep.
- Model selection algorithm.
- Keep all the features, but reduce magnitude/ value of parameters θj.
- Works well when we have a lot of features, each of which contributes a bit to predicting y.