aiacademy: 機器學習 intro

只 PO 不怎麼熟悉的主題

• YouTube

ML 1 (觀念): https://www.youtube.com/playlist?list=PL1f_B9coMEeB9vVZLAVVY99Q6jUyMxxZ- ML 2 (實作): https://www.youtube.com/playlist?list=PL1f_B9coMEeAq0pcCCMx0UoxRkQ392hy2 ML 3 (觀念 & 實作): https://www.youtube.com/playlist?list=PL1f_B9coMEeB0uxQwlKLGGyDpI_Xs8iCY ML 4 (觀念 & 實作 ): https://www.youtube.com/playlist?list=PL1f_B9coMEeCvbetNGYmW7fWUBSo0-D_i ML 5 (觀念 & 實作 ): https://www.youtube.com/playlist?list=PL1f_B9coMEeDnlocZvO4vREgupj3TWhh5 ML 6 (觀念 & 實作 ): https://www.youtube.com/playlist?list=PL1f_B9coMEeDPl3dZ_ZmoDB1A2gjPa3hg ML 7 (觀念 & 實作 ): https://www.youtube.com/playlist?list=PL1f_B9coMEeDMRTE71laJp3BYe9dOE_Ak

Gradient Descent VS. Stochastic Gradient Descent

• sgd:
  • 每次 只看一筆資料，走最好的方向

• GD VS. SGD:

  • Steps:

    • GD: fewer steps
    • SGD: more steps

  • Computation of each step

    • GD: look through all the training instances
    • SGD: look onlu one training instance

• Pros and Cons of SGD:

  • Pros:

    • When the training data is large with some (near) redundant instances, SGD is usually much faster to converge than GD

    • Supports online learning: model 比較能反映出 features 和 target variables 相對應得關係

    • Sometimes can pass local minimum

  • Cons:

    • Tends to bouncing around minimum

Close form solution

庫ㄟ! 第一次聽到 Close form solution

• 當problem 是 multiple linear regerssion的時候算 θ:

  • 可用 close form solution:

Close form solution VS. Gradient Descent

• If the number of features is small, close form solution is probably acceptable

• However, if the number of features is large, using gradient descent is more efficient

• Moreove, gradient descent is capable of solving more complex optimization problem

• in many cases, ∂J(θ)/∂θ = 0 has no closed-from solution…. , But we can still apply gradient descent