# 機器學習 - computing parameters analytically

## Normal Equation : article

 Size (feet^2) Number of bedrooms Number of floors Age of home (years) Price (\$1000) 2104 5 1 45 460 1416 3 2 40 232 1534 3 2 30 315 852 2 1 36 178
• take data set add a new column

 [x0] Size (feet^2) [x1] Number of bedrooms [x2] Number of floors [x3] Age of home (years) [x4] Price (\$1000) [y] 1 2104 5 1 45 460 1 1416 3 2 40 232 1 1534 3 2 30 315 1 852 2 1 36 178
``````example: m =4

| 1 2104 5 1 45 |        | 460 |
| 1 1416 3 2 40 |        | 232 |
x = | 1 1534 3 2 30 |    y = | 315 |
| 1  852 2 1 36 |        | 178 |

m x ( n + 1 )          m - dimensional vector

``````

• 來解釋拉:

``````  m examples (x^(1), y(1)), ..., (x^(m), y^(m)); n features

｜ x1^(i) ｜
｜ x1^(i) ｜
｜ x1^(i) ｜
x^(i) = ｜   .　  ｜ ⊂ R^n+1
｜   .　  ｜
｜   .　  ｜
｜ x1^(i) ｜

``````
``````  design matrix:
｜ (x1^(1))^T ｜
｜ (x1^(2))^T ｜
｜ (x1^(3))^T ｜
x     =    ｜     .　    ｜
(design      ｜     .　    ｜
matrix)     ｜     .　    ｜
｜ (x1^(m))^T ｜
``````
• EX:

``````             |  1     |
if x^(i) = | x1^(i) |

| 1  x1^(1) |       | y^(1) |
| 1  x1^(2) |       | y^(2) |
| 1    .    |       |   .   |
x = | 1    .    |   y = |   .   |
| 1    .    |       |   .   |
| 1  x1^(m) |       | y^(m) |

θ = ( x^T * x )^-1 * x^T * y
``````

θ = ( x^T * x )^-1 * x^T * y :

( x^T * x )^-1 is inverse of matrix x^T * x

• There is NO NEED to do feature scaling with the normal equation.

• comparison of gradient descent and the normal equation:

 `Gradient Descent` `Normal Equation` Need to choose alpha No need to choose alpha noeeds many iterations No need to iterate O (kn^2) O (k^3), need to calculate inverse of x^T * x work well when n is large slow if n is very large

## Normal Equation Noninvertibility : article

Normal equation:

θ = ( x^T * x )^-1 * x^T * y

• What if x^T * x is non-invertible ? (sigular/degenerate)

• Octave:

`````` pinv(x'*x)*x'*y
``````

What if x^T * x is non-invertible?

• Redundant features (linearly dependent)
``````Ex:
x1 = size in feet^2
x2 = sinze in m^2     (1m = 3.28 feet)
===>  x1 = (3.28)^2 * x2
``````
• Too many features (ex: m ≤ n)
• Delete some features, or use regularization

Updated: