Leetuce Be Least Sauaces, Subspacs, % Cpesit nadina gt as i el Lask time model y Xw > X Bye Wa ease rns = wee Cy cok of x i € Span x.y ws Xp) ST Ahis « sulespace ina an orthoneumal loasis for Sulo space Cvia Gace — Sct © orAlnogenalrakion ) y Ut LY be materx wf bass vectors as columns . U=- Lao ty “a Me 4. shen UU T. Zork cot of U, uy is € Span Tx, Xe Sdnm U2 XA vad wr wey nae | nee PEE =v, Xa + Ve Xort ca Fv Cau X,# Kp ke toe *V2 Coley X14 Ona hy Velen + Mem x OV ou tVe obs +t. Ween > = = ee +eR, how +0 predict label Upto ? Q ewe if vsiug orthonormal basis U ? * x recall U= XA Gece Che tn 7 ce nig nap pA Uw 2 A kaw > Gur 7S Ynew ee v> ) iS ols of X are LUnearly independent Cr=p) 2 OCX) Kus yer” So what is A ? Recall Uy = Xai. To get i, use LS Soe nee orthonormal lonsis: Given yeR , Uo from YL Chrram Scrmist) Predichon usi D compute oasis ® © = argmin hy- Uy (2 Uy @ out of sample prediction: Xneqw —> Unew } Una = wxe Vrsorem: Let XK eR, nop, X is Sal rank © cots are Lin. i eR” ( sos indee) Ged LE Ws Up be orthonormal basis vectors Son Span 3% dewey ty = Span Ts, Wed Than Y= XO where We aregnin Ny welt is given by yy UU" 4 where U=[u, 4] noe: beeause OU is an orthogeret motrin, TQ =T a LUT Pos nay 4 ye xa = XOX) Ky x “ projection vnateix" 5 or K Corijestion of yorto ned lo recall span {cots of XE = span {Uys ue 5 marae a : =? KY -KhyPR =P, = U Curvy U" 2t = + Least =a % Cussifiator Sotug: wy 7 % g n Araining Damples Cx, ydeR 2 Soe te Voc. om X en! Yi é Lung a ye Xw Xe a : £1, go] i [eb . wnAssume N2Pp, x ful le Wt ynivy a © * to ye WwW ang Vy Xwh, OLX) Ky 4 = predicted laloels on Trawivg Sam ole Xa = XOCKY'Xy =H y Predichons ° given Vt Sample wes ee baAnA ro predict Ype: TOA @® Oyen = Xnew W aa , 7 Wear fl Classi fee @ Unee > sign Xm w) € TUF Classification rule: predict al iS wow Lo -\ 8 teow <0