evaluating a learning algrithm

喔喔喔~~~ 明天第一天上課 好期待 :)

Deciding What to Try Next

• When you test your hypothesis on a new set of houses, you find that it makes unacceptably large errors in its predictions.
  What should you try next?

  • Get more training examples

  • Try smaller sets of features

  • Try getting additional features

  • Try adding polynomial featyres

  • Try decreasing λ

  • Try increasing λ
    

• Machine learning diagnostic:

  • Diagnostic: A test that you can run to gain insight what is/isn’t working with a learning algorithm, and gain guidance as to how best to improve its performance.

  • Diagnostics can take time to implement, but doing so can be very good use of your time.

  • A diagnostic can sometimes rule out certain courses of action (changes to your learning algorithm) as being unlikely to improve its performance significantly

Evaluating a Hypothesis : article

• Once we have done some trouble shooting for errors in our predicions by:

  • Getting more training examples

  • Trying smaller sets of features

  • Trying addiontal features

  • Trying polynomial features

  • Increasing or decreasing λ

• 把資料集合分成 training set 和 test set，通常比例分成 70% 和 30 %

  • The new procedure using these two sets is then:

    1. Learn Θ and minimize Jtrain(Θ) using the training set

    2. Compute the test set error Jtest(Θ)

The test set error

1. For linear regression
2. For Classification ~ Misclassification error (aka 0/1 misclassification error):

Model Selection and Train/Validation/Test Sets : article

Model Selection:

用 test set 來去抓J(θ)，結果會造成

Train / Validation / Test Sets

Train / Validation / Test Sets Error