MATH 182 : HIDDEN DATAIN RANDOM MATRICES

Todd Kemp ,APM 5202

www.math.ucsd.edu/ntkemp/182

TODAY : Linear Regression ; begin PCANEXT : PCA

Homework 2 : Due Friday , Jan 31, before midnight .

office Hours : This week, Kemp's OH

Thursday e - It a

Friday 11:30 a - 12:3op

Least-squaresAssuming Gaussian noise

,we saw that the MLE approach to

linear regression y - x. x. + b is least squares error :

argmjiyE.ly ; - hi bothamputation :

"int 's. '

- I?÷l A- I :L1. Bjtb= Ij - p = Iejtp

LHgebatthRue !we have data flag, gits! . .

We produce E- I if . 9=1%1 .Then we seek p minimizing

Sept. Hy - ¥112

Statisticallytheorem

..

The regression estimator p is consistentand unbiased

.

-

Limitation : Regression suggests the data six; , y;D,-7 ,

approximately live in a hyperplane y = a. Bt b .

Why just a codimension l l affine ) subspace ?Also,e g . y = - x, tzxz - 4

Affineswbspabst-c.IRis an affine subspace if it has the form-

A =

1¥Prep : A s Rm is an affine subspace sff

How do we specify a subspace ? Choose a basis.

Go ahead and make it an an . bas's .

Affine subspace AM

The core idea of PCA is to find the " best fit affine subspace "for the given data 34,3k

,-→ In .

Ie. find a translation vector pie and a subspace V s . t

.

x.j -y En V

Least-squares :Mo,Q o

Besttranslationveckerted( p , Q , pi . . . ., I = Haj -p - Q pill?

Find the minimum @ critical point (D smooth convex function)start a µ .

Take the directional derivative

da, Ted ( pot te ,Q , fi. . ..Htt .- o

Special Case : m=2,

d -- I

⇒ = I Q = is c- HE unit vector

I pje.IR'

x. ja In t Apj- assume we've already found I

.