ulmenslonallLy

8educuon
Mouvauon l:
uaLa Compresslon
Machlne Learnlng
Andrew ng
!"#" %&'()*++,&-
(
l
n
c
h
e
s
)

(cm)
8educe daLa from
2u Lo 1u
Andrew ng
!"#" %&'()*++,&-
8educe daLa from
2u Lo 1u
(
l
n
c
h
e
s
)

(cm)
Andrew ng
!"#" %&'()*++,&-
8educe daLa from 3u Lo 2u
ulmenslonallLy
8educuon
Mouvauon ll:
uaLa vlsuallzauon
Machlne Learnlng
Andrew ng
!"#" .,+/"0,1"2&-
[resources from en.wlklpedla.org]
CounLry
Cu
(Lrllllons of
uS$)
er caplLa
Cu
(Lhousands
of lnLl. $)
Puman
uevelop-
menL lndex
Llfe
expecLancy
overLy
lndex
(Clnl as
percenLage)
Mean
household
lncome
(Lhousands
of uS$) .
Canada 1.377 39.17 0.908 80.7 32.6 67.293 .
Chlna 3.878 7.34 0.687 73 46.9 10.22 .
lndla 1.632 3.41 0.347 64.7 36.8 0.733 .
8ussla 1.48 19.84 0.733 63.3 39.9 0.72 .
Slngapore 0.223 36.69 0.866 80 42.3 67.1 .
uSA 14.327 46.86 0.91 78.3 40.8 84.3 .
. . . . . . .
Andrew ng
!"#" .,+/"0,1"2&-
CounLry
Canada 1.6 1.2
Chlna 1.7 0.3
lndla 1.6 0.2
8ussla 1.4 0.3
Slngapore 0.3 1.7
uSA 2 1.3
. . .
Andrew ng
!"#" .,+/"0,1"2&-
ulmenslonallLy
8educuon
rlnclpal ComponenL
Analysls problem
formulauon
Machlne Learnlng
Andrew ng
3),-4,("0 %&'(&-*-# 5-"06+,+ 73%58 ()&90*' :&)'/0"2&-
Andrew ng
3),-4,("0 %&'(&-*-# 5-"06+,+ 73%58 ()&90*' :&)'/0"2&-
8educe from 2-dlmenslon Lo 1-dlmenslon: llnd a dlrecuon (a vecLor )
onLo whlch Lo pro[ecL Lhe daLa so as Lo mlnlmlze Lhe pro[ecuon error.
8educe from n-dlmenslon Lo k-dlmenslon: llnd vecLors
onLo whlch Lo pro[ecL Lhe daLa, so as Lo mlnlmlze Lhe pro[ecuon error.
Andrew ng
3%5 ,+ -&# 0,-*") )*;)*++,&-
Andrew ng
3%5 ,+ -&# 0,-*") )*;)*++,&-
ulmenslonallLy
8educuon
rlnclpal ComponenL
Analysls algorlLhm
Machlne Learnlng
Andrew ng
1ralnlng seL:
reprocesslng (feaLure scallng/mean normallzauon):
!"#" ()*()&4*++,-;
8eplace each wlLh .
lf dlerenL feaLures on dlerenL scales (e.g., slze of house,
number of bedrooms), scale feaLures Lo have comparable
range of values.
Andrew ng
3),-4,("0 %&'(&-*-# 5-"06+,+ 73%58 "0;&),#<'
8educe daLa from 2u Lo 1u 8educe daLa from 3u Lo 2u
Andrew ng
3),-4,("0 %&'(&-*-# 5-"06+,+ 73%58 "0;&),#<'
8educe daLa from -dlmenslons Lo -dlmenslons
CompuLe covarlance maLrlx":


CompuLe elgenvecLors" of maLrlx :
[U,S,V] = svd(Sigma);
Andrew ng
3),-4,("0 %&'(&-*-# 5-"06+,+ 73%58 "0;&),#<'
lrom , we geL: [U,S,V] = svd(Sigma)
Andrew ng
3),-4,("0 %&'(&-*-# 5-"06+,+ 73%58 "0;&),#<' +/''")6
Aer mean normallzauon (ensure every feaLure has
zero mean) and opuonally feaLure scallng:
Sigma =

[U,S,V] = svd(Sigma);
Ureduce = U(:,1:k);
z = Ureduce*x;
ulmenslonallLy
8educuon
8econsLrucuon from
compressed
represenLauon
Machlne Learnlng
Andrew ng
=*4&-+#)/42&- :)&' 4&'()*++*> )*()*+*-#"2&-
ulmenslonallLy
8educuon
Chooslng Lhe number of
prlnclpal componenLs
Machlne Learnlng
Andrew ng
%<&&+,-; 7-/'9*) &: (),-4,("0 4&'(&-*-#+8
Average squared pro[ecuon error:
1oLal varlauon ln Lhe daLa:

1yplcally, choose Lo be smallesL value so LhaL




99 of varlance ls reLalned"
(1)
Andrew ng
%<&&+,-; 7-/'9*) &: (),-4,("0 4&'(&-*-#+8
AlgorlLhm:
1ry CA wlLh
CompuLe

Check lf
[U,S,V] = svd(Sigma)
Andrew ng
%<&&+,-; 7-/'9*) &: (),-4,("0 4&'(&-*-#+8
[U,S,V] = svd(Sigma)
lck smallesL value of for whlch



(99 of varlance reLalned)
ulmenslonallLy
8educuon
Advlce for
applylng CA
Machlne Learnlng
Andrew ng
?/(*)@,+*> 0*")-,-; +(**>/(
LxLracL lnpuLs:
unlabeled daLaseL:
new Lralnlng seL:
noLe: Mapplng should be dened by runnlng CA
only on Lhe Lralnlng seL. 1hls mapplng can be applled as well Lo
Lhe examples and ln Lhe cross valldauon and LesL
seLs.
Andrew ng
5((0,4"2&- &: 3%5
- Compresslon
- 8educe memory/dlsk needed Lo sLore daLa
- Speed up learnlng algorlLhm
- vlsuallzauon
Andrew ng
A"> /+* &: 3%5B C& ()*@*-# &@*)DE-;
use lnsLead of Lo reduce Lhe number of
feaLures Lo
1hus, fewer feaLures, less llkely Lo overL.
1hls mlghL work Ck, buL lsn'L a good way Lo address
overmng. use regularlzauon lnsLead.
Andrew ng
3%5 ,+ +&'*2'*+ /+*> F<*)* ,# +<&/0>-G# 9*
ueslgn of ML sysLem:
- CeL Lralnlng seL
- 8un CA Lo reduce ln dlmenslon Lo geL
- 1raln loglsuc regresslon on
- 1esL on LesL seL: Map Lo . 8un on

Pow abouL dolng Lhe whole Lhlng wlLhouL uslng CA?
8efore lmplemenung CA, rsL Lry runnlng whaLever you wanL Lo
do wlLh Lhe orlglnal/raw daLa . Cnly lf LhaL doesn'L do whaL
you wanL, Lhen lmplemenL CA and conslder uslng .