Tags:

ML 1 (觀念): https://www.youtube.com/playlist?list=PL1f_B9coMEeB9vVZLAVVY99Q6jUyMxxZ- ML 2 (實作): https://www.youtube.com/playlist?list=PL1f_B9coMEeAq0pcCCMx0UoxRkQ392hy2 ML 3 (觀念 & 實作): https://www.youtube.com/playlist?list=PL1f_B9coMEeB0uxQwlKLGGyDpI_Xs8iCY ML 4 (觀念 & 實作 ): https://www.youtube.com/playlist?list=PL1f_B9coMEeCvbetNGYmW7fWUBSo0-D_i ML 5 (觀念 & 實作 ): https://www.youtube.com/playlist?list=PL1f_B9coMEeDnlocZvO4vREgupj3TWhh5 ML 6 (觀念 & 實作 ): https://www.youtube.com/playlist?list=PL1f_B9coMEeDPl3dZ_ZmoDB1A2gjPa3hg ML 7 (觀念 & 實作 ): https://www.youtube.com/playlist?list=PL1f_B9coMEeDMRTE71laJp3BYe9dOE_Ak

• sgd:
• 每次 只看一筆資料，走最好的方向
• GD VS. SGD:

• Steps:

• GD: fewer steps
• SGD: more steps
• Computation of each step

• GD: look through all the training instances
• SGD: look onlu one training instance
• Pros and Cons of SGD:

• Pros:
• When the training data is large with some (near) redundant instances, SGD is usually much faster to converge than GD

• Supports online learning: model 比較能反映出 features 和 target variables 相對應得關係

• Sometimes can pass local minimum

• Cons:

• Tends to bouncing around minimum

Close form solution

• 當problem 是 multiple linear regerssion的時候算 θ:

• 可用 close form solution:

Close form solution VS. Gradient Descent

• If the number of features is small, close form solution is probably acceptable

• However, if the number of features is large, using gradient descent is more efficient

• Moreove, gradient descent is capable of solving more complex optimization problem

• in many cases, `∂J(θ)/∂θ = 0` has no closed-from solution…. , But we can still apply gradient descent

Tags:

Updated: